Rank as an Inherent incentive: Experimental Evidence from a Cognitively Challenging Task

Transcription

1 Rank as an Inherent incentive: Experimental Evidence from a Cognitively Challenging Task Tony So University of Auckland Abstract We study the impact of relative performance feedback as an inherent incentive mechanism to enhance productivity in a cognitively challenging task. In each of multiple rounds subjects are shown two cue values, Cue A and Cue B, and asked to predict the value of a third variable X, which is a function of the two cue values. We use forecast errors, the absolute difference between the predicted value of X and the actual value of X, as the metric for performance. Our treatments include: (1) piece rates, where subjects are paid on the basis of only their own absolute errors; (2) piece-rate-win-lose, where subjects are paired and paid a piece rate that depends on their own absolute errors only, but informed about whether they did better or worse than their partners; (3) a two-person winner-take-all-tournament where subjects are paired and the one with the smaller error earns a positive payoff while the other earns nothing. We find that average forecast errors are smaller in the piece-rate-win-lose treatment, compared to the piece-rate and the winner-take-all-tournament treatments, with no difference between the last two. JEL Codes: C91; J24, J33, J39 Keywords: Experiments; Payment schemes; Tournaments; Piece-rates; Productivity; Learning 1

2 Preface This paper is based upon my recently submitted PhD thesis at the University of Auckland titled: Essays in Behavioural Labour Economics, supervised by Professor Ananish Chaudhuri and Dr Ryan Greenaway-McGrevy. The thesis analyses how different pay schemes and the provision of relative performance feedback affects worker performance. This paper focuses on the notion of tournament decomposition separating the effects of competition for rank and competition for payoffs, both of which may play a part in motivating performance under tournaments. As will be discussed in greater detail in this paper, competition for rank refers to people s desire to compete in order to achieve favourable comparisons with others, while competition for payoffs refers to the desire to compete for the monetary prizes which are associated with how people rank within the tournament. 2

3 1. Introduction A fundamental question in labour economics relates to how workers are paid and how productivity is affected by the choice of payment schemes. There is a vast literature looking at the motivational aspects of various pay schemes. See Prendergast (1999) for a review. Conventional wisdom in economics suggests that incentive schemes that pay on the basis of performance, such as output-dependent piece rates, are effective in inducing better performance (for example, see Lazear (2000)). Piece rates, however, are inapplicable in situations where output cannot be easily observed or measured. In these instances, an employer could implement tournament pay schemes as they do not rely on absolute performance. Rank-order tournaments pay workers a fixed amount depending on how their performance ranks amongst others, with the prize monotonically increasing with rank such that those with a higher rank receive higher pay. Theoretical analyses of tournaments (Lazear & Rosen, 1981; Green & Stokey, 1983; Nalebuff & Stiglitz, 1983) show that they are effective in eliciting output at a level analogous to piece rates. This result is borne out by results in a laboratory experiment by Bull, Schotter, and Weigelt (1987), where they show that, on average, numerical effort choices made under tournaments are statistically not different than those under piece rates, though variance of effort choices under tournaments are larger. There is empirical evidence suggesting that the rank-dependent incentives of tournaments positively influence behaviour in various field settings such as corporate promotions and sporting competitions (for example see Eriksson, 1999; Ehrenberg & Bognanno, 1990; Taylor & Trogdon, 2002). However, while prior work in the area has compared tournaments with piece rates, one issue has not received enough attention. In moving from piece-rates to tournaments, the nature of the competition and the inherent incentives change with two distinct components to that change. The first component and the more obvious one arises from the fact that in a tournament players are competing for a higher payoff, which we will refer to as competing for payoffs; in a piece rate, payoffs depend on only on one s own performance while in a rank-order tournament it depends on one s rank. If the tournament happens to be of a winnertake-all type, then coming second implies zero monetary payoff. But there is a second component to this change, since, in a tournament, agents must outperform their peers in order to attain a higher rank. While a higher rank may correspond to 3

4 a higher tangible reward (such as promotion tournaments), agents may simply be motivated by the higher rank itself, in the sense that they derive pleasure or pain from the act of winning or losing respectively (as in a friendly game of tennis, squash or chess). 1 We will refer to this loosely as competing for rank. There is ample evidence that information about one s relative rank, vis-à-vis one s peers, has a positive impact on performance, even when that higher rank does not translate into higher monetary payoffs. We provide a brief overview of this literature in Section 2 below. In this paper we study the distinction between the notions of competing for rank and competing for payoffs and their impact on performance in a multiple cue probabilistic learning (MCPL) task introduced by Brown (1995, 1998). This is a cognitively challenging task where in each of multiple rounds subjects are shown two cue values and asked to predict the value of an unknown variable, which is a function of those two cue values. The cue values shown to subjects change from one round to the next but the underlying function does not. Subjects do not know what the underlying functional form is but they do know that this function remains unchanged in every round. The goal for the subjects is to make accurate predictions on the basis of the cue values shown to them in each round, where accuracy is measured by the forecast error the absolute deviation of their predicted value from the actual value of the variable in that round. Henceforth, we will refer to forecast errors, rather loosely, as productivity, in the sense that smaller errors represent higher productivity and vice versa. We implement three different treatments: (1) piece rates, where subjects are paid on the basis of their own forecast errors; (2) piece-rate-win-lose, where subjects are paired and paid a piece rate that depends on their own absolute errors only, but informed about whether they did better or worse than their partners; (3) a two-person winner-take-all-tournament where subjects are paired and the one with the smaller error earns a positive payoff while the other earns nothing. Our primary aim is to understand which one of these treatments leads to the lowest forecast errors. We explain our design in greater detail in Section 3 below. In summary, we will show the following. Average productivity is highest (and forecast errors smallest) under the piecerate win-lose treatment where subjects get paid a piece rate but are also provided with rank information even though the latter does not impact their earnings. Average productivity in 1 The idea behind rank competition in our study is similar to what Kräkel (2008) describes as emotions, where positive and negative emotions are derived from winning and losing respectively. 4

5 this treatment is higher (and average errors smaller) compared to that under a piece rate and a winner take-all tournament. Echoing the results of Bull et al. (1987), average productivity is not different under piece rates or tournaments. Our results suggest that, in the MCPL task that we implement, competing for rank provides better incentives than competing for payoffs. We proceed as follows. In Section 2 we provide a brief overview of the related literature that looks at the impact of rank information on productivity. In Section 3 we explain our experimental design and present several hypotheses. In Section 4 we discuss our findings and finally in Section 5 we make some concluding comments. 2. Impact of Rank Information As noted above, one thing that changes in going from a piece rate scheme to a tournament is that subjects now get to see how they are performing vis-à-vis their peers. There is now a large literature which suggests that information about rank even if that rank does not affect monetary payoffs has implications for worker productivity. Mas and Moretti (2009) show that supermarket checkout operators who are paid fixed wages became more productive when, with a change in shifts, the average productivity of their peers increase. This effect was found to be attributable to workers who were directly in the line of sight of co-workers, implying that agents productivity improved when they were being observed by others. In other words, peer pressure is found to have a motivating effect. Productivity spill-overs and the incentive to free-ride off the effort of fellow workers are key features of the Mas and Moretti (2009) study. Falk and Ichino (2006) look into peer effects in the absence of productivity spill-overs. They analyse levels and variance of performance when students were paid a flat wage to stuff envelopes. In the control treatment, subjects work alone while in an experimental treatment they work on their own but in the presence of a second subject located in the same room. In the latter setup, subjects are able to observe each other s output. The authors find that performance (in terms of number of enveloped stuffed) was higher in the experimental treatment where subjects work in pairs. Furthermore, variation of output within pairs is lower than between pairs, indicating that peer effects mutually motivated each individual to perform at a level similar to her partner. Blanes i Vidal and Nossol (2011) assess the effect of relative performance feedback on the performance of German warehouse workers, who are paid a piece rate on top of a base 5

6 salary. 2 Here the workers were notified two months in advance that they would be receiving additional rank information in their payslips. The revelation of rank information was found to have a positive effect on productivity. Moreover, a distinct positive effect on productivity exists even prior to the relative rank information being revealed as long as subjects are aware that it is impending. In other words, both the anticipation and revelation of relative feedback has a motivating effect on performance. In Kuhnen and Tymula (2012) participants solve multiplication problems over a number of timed rounds and are paid a fixed salary for their participation. In one treatment participants receive relative performance feedback while in a second they receive such feedback with 0.5 probability while in a third treatment no feedback is provided. Players in both certain feedback and probabilistic feedback performed better than those who did not receive feedback while there are no differences in performance in the first two treatments. It appears that while feedback matters even the likelihood of receiving feedback can serve as a motivating force. Cadsby, Engle-Warnick, Fang and Song (2010) run experiments with Chinese students and Chinese factory workers, where participants are asked to add five 2-digit numbers over multiple rounds. The study has a 2 x 2 design which varies payment type rank-order tournament or fixed salary and the nature of feedback after each round public or private. Students paid under a tournament scheme perform better when the rank information is public while the nature of feedback had no effect for factory workers. Two papers that fail to find a positive impact of peer pressure are Eriksson et al. (2009) and Bellamare et al. (2010). Eriksson et al. s subjects undertake a number adding task like Cadsby et al. while the feedback is either continuous (i.e., up to date feedback is available at every point in time) or discrete (provided once at the half way mark). There is a third treatment with no feedback. Eriksson et al. (2009) found no difference in the number of correct answers between participants who were provided either discrete or continuous feedback compared to participants who do not receive relative feedback at all. Eriksson et al. suggest that performance was high in the no feedback treatment and hence there was not much scope for upward movement in the feedback treatments which may explain the lack of an effect. 2 In the literature it is usually assumed that payment schemes that combine fixed salaries with piece rates have the same incentive properties as that of the latter. 6

7 In Bellemare et al. (2010) subjects undertake a data-entry task and are paid either a fixed wage or a piece rate. Bellemare et al. (2010) report an inverse U-shaped impact of peer pressure for men under fixed wages in that extremely low and extremely high levels of peer pressure affected performance adversely. But, by and large, the provision of relative performance feedback had little impact on performance. However, as opposed to other studies where relative information is provided in real time with all subjects performing the task concurrently, Bellemare et al. provide relative feedback by comparing the behaviour of a current participant with that of someone who took part in the experiment on a previous occasion. It is possible that this design feature made the relative performance feedback less salient for subjects. Two further papers find mixed evidence regarding the effect of providing relative feedback. Both Hannan, Krishnan and Newman (2008) and Azmat and Iriberri (2016) find that the provision of relative performance feedback has different effects depending on how people are paid. In Hannan et al. (2008) subjects participate in a stylised profit maximisation problem where they took the role of a manager and selected the output level in order to maximise profit under uncertainty. The manager was paid according to either a tournament bonus or a piece rate commission on profit earned. For each of these two pay schemes, subjects in different treatments either received feedback about their rank or not. The feedback, when provided, could be either coarse or fine. The former notifies the subject whether the profit points earned is above or below the median, while the latter is more precise indicating their performance decile. Hannan et al. find that the firm s profits are higher under piece rates when relative feedback is provided than when it is not. On the other hand, the provision of feedback has no effect in tournaments when relative feedback is coarse and actually worsens earnings when fine feedback is provided. In an arithmetic task, Azmat and Iriberri (2016) look at the effect of relative feedback on piece rates and flat salaries. Relative feedback improves performance only under piece rates, while having no effect whatsoever under salaries. Finally, Azmat and Iriberri (2010) and Tran and Zeckhauser (2012) use natural data to examine the effect of relative performance feedback. Azmat and Iriberri (2010) use data from for Spanish high school students to understand whether providing relative rank information leads to improved student achievement. In the academic year , due to exogenous changes, student report cards provided information about the average class grade alongside their own grade. This resulted in students attaining higher grades that year compared to previous and subsequent 7

8 years where no such relative feedback was provided. Similarly, Tran and Zeckhauser (2012) found that Vietnamese English-language students who were notified of how they were ranked within their class, performed better than the control group who were not provided such information. It made no difference whether feedback was made publicly to all members of the class or privately to individual students. 3. Experimental Design 3.1. Task Our experiment is based on a multiple cue probabilistic learning (MCPL) task, where in each one of 20 rounds (t) participants are required to predict the value of a variable Xt based on the observation of two numerical cues provided to them 3. This task is representative of real life tasks. For example, the variable Xt can be thought of as the underlying price of a stock; the cues as variables that affect the value of a stock; and the task at hand as one of forecasting stock prices. The actual stock value is determined by the underlying equation: where Xt is the actual stock value participants are required to predict, Cue At and Cue Bt are the values of the two numerical cues provided to the participant, and εt is a random variable with uniform distribution drawn from the set [-5, 5] in each round t. Participants do not know about the error term, the exact relationship between the cue values and the value of X, or even whether the relationship is linear or non-linear. They do know, however, that while the cue values change from one round to the next, the underlying relationship does not change. We implement two variants of the task. In the Single Cue task, Cue A is fixed at the value of 150 for each of the 20 rounds, while Cue B changes each round. This is designed to be less difficult than the Dual Cue task, where both cue values change in each round. For both tasks, the sequence in which the cue values appear from one round to the next, is identical across treatments. Table 1 shows the cue values and corresponding stock prices for each round. 3 MCPL tasks are commonly used in psychology to study learning (see Balzer, Doherty & O Connor, 1989 for a review). In economics, besides Brown (1995, 1998), this task has been used by Vandegrift and Brown (2003), Vandegrift, Yavas, and Brown (2007) as well. 8

9 Our metric of performance is the absolute forecast error, the absolute distance between the predicted value (Xt P ) and the actual value Xt *, ( Xt P - Xt * ). Forecast errors measure the accuracy of the predicted value, so smaller errors imply more accurate forecasts and therefore better performance (and higher productivity). Absolute errors are an appropriate measure of performance as they reflect the cognitively challenging nature of the MCPL task. 4 Table 1: Actual Cues and Stock Prices Single Cue Task Dual Cue Task Round Stock Stock Cue A Cue B Cue A Cue B Price Price Treatments We report on data from three treatments for the purposes of this paper, which is part of a larger study involving other treatments and analyses. These three treatments are: (1) Piece Rate (PR), (2) Piece Rate Win-Lose (PRWL); and (3) two-person Winner-Takes-All Tournament (WTAT). The treatment intervention comes into play from round 6. 4 As will be clear from Table 1, the MCPL task is difficult enough even when a single cue is changing. When both cues change the degree of difficulty increases. Subjects figure out relatively quickly that the actual stock price is a convex combination of the two cue values and therefore get reasonably close to the actual value when the cue values are close but the errors get large when the cue values are far apart. 9

10 In the first five rounds of each of these treatments, subjects are paid piece rates according to their individual performance. At this point, subjects do not know which treatment they are assigned to, but do know that treatment interventions might or might not take place prior to the sixth round. Since these rounds are identical in each of the treatments, we are able to use performance across these rounds to proxy for subjects underlying ability pertaining to the task. It is important to control for ability given that our MCPL task is relatively difficult and given the possibility of significant heterogeneity in ability levels across subjects. The piece rate that takes place across rounds 1 to 5 in each of the treatments pays participants NZ $1 minus their forecast error for the round. For example, if the absolute error is 20 then the payment for that round is NZ $0.80. If the forecast error is greater than 100, then earnings for that round is set to zero. Here subjects aim to minimise their forecast error in each round, which in turn will lead to higher payoff. From round 6 onwards, treatment interventions take place. In the PR treatment, subjects continue to earn piece rates on their individual performance across rounds 6 to 20, and subjects are instructed accordingly. Figure 1A presents a screenshot to show what the subjects get to see at the end of a round. This is the information that a subject will be looking at prior to the beginning of round 10. The PRWL treatment adds rank competition on top of the PR treatment. Here, starting with round 6, subjects are paired, with random re-matching of pairs between rounds. They are paid according to their own forecast errors in each round, i.e. the payment scheme is the same piece rate. Furthermore, from round 6 onwards, in each round the subjects are also told whether they have Won or Lost depending on whether a particular subject s error was smaller than or larger than her pair member respectively. However, whether a subject won or lost a particular round has no bearing on her earnings for that round since each subject continues to get paid on the basis of her own forecast errors. The rank information is simply designed to capture positive or negative emotions from winning or losing respectively. Figure 1B shows a screenshot of this treatment. 10

11 Figure 1A: Screenshot for PR treatment Round Cue A Cue B Your Forecast Actual Price Forecasting Error Earnings this round Round number Player name Player ID Cue A $ $0.50 Cue B $ $0.77 Enter Forecast $ Tso $0.64 SUBMIT RESET $ $ $0.96 Figure 1B: Screenshot for PRWL treatment Round Cue A Cue B Your Actual Forecasting Earnings WIN Round number Forecast 75 Price 69 Error 6 this round $0.94 or LOSE --- Player name Player ID Cue A $ Cue B $ $ Enter Forecast $ $0.64 WIN SUBMIT RESET $0.82 WIN $0.80 LOSE $0.96 WIN 10 Tso Figure 1C: Screenshot for WTAT treatment 11

12 Round Cue A Cue B Your Actual Forecasting Earnings WIN Round number Forecast 75 Price 69 Error 6 this round 0.94 or LOSE --- Player name Player ID Cue A Cue B Enter Forecast Tso $1 WIN SUBMIT RESET $1 WIN $0 LOSE $1 WIN The WTAT treatment also starts in round 6, following five rounds of piece rate payments. As in the PRWL treatment, from round 6 onwards, we form subjects into pairs (with random re-matching from one round to the next), except here we implement a winnertakes-all tournament scheme, where in each round the subject with the smaller absolute error wins NZ $1 while the subject with the larger error gets zero. 5 Figure 1C presents a screenshot. Compared to the PRWL treatment, from round 6 onwards, the WTAT treatment not only provides the win/lose information but changes the payoffs as well. So the WTAT treatment incorporates payoff competition on top of the rank competition in the PRWL treatment Research Questions and Hypotheses In this section we formulate some hypotheses that will guide our analysis later in the paper. We start by comparing our two pay schemes: piece rates and tournaments. Theory predicts that the level of effort exerted under tournaments are identical to those under piece rates, a finding confirmed empirically by Bull et al. (1987). We shall refer to this as Piece Rate Equivalence. As such, we would expect the PR and WTAT treatments to perform similarly and have similar forecast errors. This leads to our first hypothesis. H1. Piece Rate Equivalence holds, where the productivity of tournaments are similar to that of piece rates. This implies: PR(Errors) WTAT(Errors). 5 If the forecast errors of a particular pair are equal in particular round, then the tie is broken by randomisation. 12

13 Next we look at how rank competition affects productivity. In order to address this we compare the PR and PRWL treatments. Both treatments are identical, except that the latter provides additional information at the end of each round about whether subjects performed better or worse than their partners. Subjects are paid piece rates in both treatments; the rank information does not impact earnings, allowing us to isolate the effect of rank competition from payoff competition. If people are motivated solely by earnings, then we would not expect rank competition to have any effect on productivity. 6 On the other hand if people derive utility (disutility) from winning (losing) a particular round, then we would expect them to work harder to improve (reduce) their chances of winning (losing) improving their forecast errors. This effect may consist of an ex-ante anticipation effect (Blanes i Vidal & Nossol, 2011) that may be associated with preferences for status and respect (see Ellingsen & Johannesson, 2007 for a review), or an ex-post revelation effect when people react to the feedback received. Either of these effects or both, would lead better performance in the PRWL treatment compared to the PR treatment. This leads to our second hypothesis. H2. The provision of relative performance feedback improves productivity. This implies: PRWL (Errors) < PR (Errors). We now turn to the issue of competing for payoffs by comparing the PRWL and WTAT treatments. Both treatments feature rank feedback, but differ in terms of incentives: piece rates and rank-dependent prizes respectively. From Hypothesis H1, we expect tournaments to perform similarly to piece rates according to Piece Rate Equivalence. Does this equivalence still hold with our rank-augmented piece rates? If piece rates perform no differently to tournaments (H1), and rank competition induces higher performance from players (H2), then we would accordingly expect the feedback-augmented piece rates (PRWL) to perform better than tournaments (WTAT). Eriksson et al. (2009) provide evidence indicative of this; piece rates with relative feedback perform better than tournaments featuring identical feedback. This suggests that the rank- 6 Under a principal-agent framework, Aoyagi (2010) studies the effect of the principal providing feedback about the intermediate performance gap to agents in a dynamic tournament. He finds that the effect this feedback has on the effort exerted by agents depends on the nature of their cost of effort functions. If agents marginal cost of effort are concave (convex), then no (full) feedback induces low effort from agents. If marginal cost is linear, then effort is not affected by feedback policy. We caution applying the insights of Aoyagi to our study of rank competition, since the dynamics will be different according to the different setup (multi-stage tournaments in Aoyagi; finitely repeated single stage tournament here) and pay schemes (tournaments in Aoyagi; piece rates in our study). 13

14 dependent payoffs inherent in tournaments do not perform as well as piece rates do once rank competition has been controlled for. It is the motivating effect of rank competition that underpins Piece Rate Equivalence. This leads us to our third hypothesis. H3. When relative feedback is added to piece rates, this will lead to better performance than in tournaments. This implies PRWL (Errors) < WTAT (Errors). There are at least two reasons why tournament performance may be worse than that under piece rates with rank feedback. First, there is an element of uncertainty in rank payoffs. In order to get a payoff, one has to perform better than one s peers while not knowing how others will perform due to random rematching of partners and since players are only informed of whether they have won or lost, rather than on the margin of winning or losing. Under winner-takes-all tournaments, the possibility of losing means that the exertion of costly effort might not yield any return. For a given degree of risk aversion a person would likely exert less effort under a tournament than a piece rate, for which payoffs are deterministic. The second reason why tournament performance may suffer relates to the concept of shying away from competition (Niederle & Vesterlund, 2007). Those averse to competition may find the tournament set-up challenging. While there is an element of competition in both PRWL and WTAT, since relative performance feedback is provided in both, competition is more salient in WTAT since payoffs are directly linked to rank. Competition avoidance could further explain why productivity may be lower in WTAT than in PRWL. While we cannot explicitly observe instances of this, we control for competition-aversion through trait anxiety scores, which are elicited before participants learn of the task. See Segal and Weinberg (1984) for a discussion on how trait anxiety can serve as a proxy for competition avoidance. To summarize, our three hypotheses jointly predict forecast errors would be such that PRWL (Errors) < PR (Errors) WTAT (Errors). Taken together this suggests that with the provision of rank feedback, piece rates would perform better than tournaments, meaning that competing for rank provides a stronger incentive than competing for payoffs. 14

15 3.4. Experimental Procedure Sessions were conducted at a computer lab in our University using primarily first year students in commerce. We report data obtained from 236 participants in the three treatments described above. Participants are seated at computer cubicles with privacy partitions and are cautioned about not communicating with any other subject. To start with, participants are asked to fill out a questionnaire which elicits participants trait anxiety level. See Spielberger, Gorsuch, Lushene, Vagg and Jacobs (1983). This is shown in the appendix. The questionnaire consists of 20 questions that are answered on a 1 to 4 scale. Questions 1, 6, 7, 10, 13, 16 and 19 are reverse scored. The questionnaire is designed to measure a subject s general tendency to feel anxious rather than their current level of anxiety (McNaughton, 2011). A higher score generated from the pre-task questionnaire indicates a higher level of trait anxiety associated with the individual. We use trait anxiety as a proxy of each subject s competitive preferences, so we can account for potential dropouts as a result of shying away from rank or feedback competition (Niederle and Vesterlund, 2007). Segal and Weinberg (1984) find that trait anxiety is higher among women than men which is attributed to differences in competitive preferences. Following this we hand out the instructions for the forecasting task. The instructions are read out loud after subjects have had a chance to read them privately on their own. The appendix contains a copy of the instructions. These instructions describe the task, inform players that they will be earning piece rates in the first five rounds, and that there might be a change in the way the game is played in rounds 6 to 20. The instructions are identical for people in different treatments. They are told that they will be provided further information prior to the start of round 6. Subjects are also provided with ten examples for Cue A, Cue B and X. This is shown in Table 2. Table 2: Cue Values given to subjects as practice examples Single Cue Task Dual Cue Task Cue A Cue B Actual Actual Cue A Cue B Price Price

16 In the PR treatment, following round 5, they are told that there are no further instructions and they should continue as before. In the PRWL treatment, after round 5, they are told that in going forward they will be paired with another player in each round, and informed of whether they won or lost a round. They are also told that this rank information has no bearing on their earnings, which still depend only on their absolute errors in any given round. In the WTAT treatment they are told both about the pairing and that from round 6 onwards they will earn either $1 or nothing in each round. In both the PRWL and WTAT treatments subjects are told that they will be randomly re-matched from one round to the next. At the conclusion of the task, participants are asked to fill out a post-task questionnaire, which elicits information about participants intrinsic motivation, including self-reports of how competent they felt at the task, how motivated they were, how interesting they found the task, how much effort they exerted and how close to they felt to other participants in the room. We also collected basic demographic information including gender, age and ethnicity. We do not elaborate on the psychological questionnaires since we do not exploit data from them for the purposes of this study. After filling out the questionnaire, participants are privately paid their earnings from the 20 rounds of the forecasting task in cash, plus a $5 show-up fee. On average, participants earned $20 in each treatment Results Table 3 provides a broad overview of the forecast errors in different treatments along with the number of subjects and sessions in each treatment. 8 Not surprisingly the errors are 7 Under tournaments, the expected earnings in a round is $0.50 (50% chance of winning the $1 prize), which is much lower than the ex-post average earnings under piece rates in the PR and PRWL treatments. In the WTAT treatment, we paid participants an extra $4 to calibrate their average earnings with other treatments. 8 We had 36 subjects in the PRWL treatment but due to reasons beyond our control, one subject left early. Since we needed to put subjects in pairs from round 6 onwards, we discreetly replaced the departing subject with one of our experimental assistants (who had no prior experience with the game). We have excluded the choices made by this subject (and the replacement) from our analysis. However, given the random re-matching of subjects and the fact that subjects never get to see the ID numbers of their partners, we have retained the data for the remaining 35 subjects. 16

17 much smaller in the single cue task than the dual cue one. What is noticeable is that in both the single and dual cue tasks, average forecast errors are highest in the WTAT treatment, followed by the PR treatment. Errors are smallest in the PRWL treatment. To explore these issues more rigorously we turn to regression analysis next. As noted before forecast errors serve as our metric for performance (= Predicted stock value Actual stock value ). Smaller forecast errors indicate higher productivity. In the first three columns of Table 4 we present random effects regressions with forecast errors as the dependent variable, with robust standard errors clustered by individual subjects. In running these regressions, we pool the data from the single cue and dual cue tasks. We use a random effects specification because we have both time-varying and time-invariant variables among our regressors, where the time-invariant regressors are inestimable under fixed effects regressions. In Column 4 of Table 4, we also present results of a quantile regression as a robustness check. 17

18 Table 3: Average errors across treatments Treatments Session # Single Cue Dual Cue Pooled Data Average Session Average N N N Error # Error Average Error Piece Rate (PR) Piece Rate Win-Lose (PRWL) Winner Take All Tournament (WTAT) Total Recall that in all treatments subjects play the PR treatment for the first 5 rounds and the treatment (if any) is implemented only at the start of Round 6. Hence in running these regressions we use data for rounds 6 through 20. In our simplest regression model, we regress forecast errors against two dummy variables which represent the PRWL and WTAT treatments (with the PR treatment serving as the reference category), as well as a linear time trend denoted by Round which captures learning over time. Our second regression model includes additional controls of each subject s gender (= 1 for women and 0 for men) and trait anxiety score. Our third regression model builds on the second, but better controls for learning by allowing time trends to differ across treatments. Our fourth regression model shares the same specification as Model 2, but instead of being estimated with random effects, is estimated with a quantile regression. The bottom of Table 4 presents a pairwise Wald test for each regression model comparing the performance of the PRWL and WTAT treatments. 18

19 Table 4: Pooled regressions for forecast errors; columns (1) to (3) present results for random effects regressions while column (4) presents results for a quantile regression Dependent variable = forecast errors = Predicted value Actual Value Independent variables Model 1 Model 2 Model 3 Model 4 Pooled Pooled Pooled Pooled Random Effects Random Effects Random Effects Quantile Regression PRWL WTAT (2.186) [0.385] (2.740) [0.454] Female --- Trait anxiety --- Round (0.075) [0.070] (2.425) [0.070] (2.958) [0.865] (1.928) [0.000] (0.150) [0.281] (0.082) [0.049] PR X Round PRWL X Round WTAT X Round Constant (1.878) [0.000] (6.164) [0.077] (3.200) [0.050] (3.390) [0.358] (1.929) [0.000] (0.150) [0.281] (0.146) [0.331] (0.153) [0.984] (0.119) [0.004] (6.630) [0.108] (0.743) [0.076] (0.750) [0.556] (0.610) [0.000] (0.044) [0.052] (0.070) [0.182] (2.052) [0.004] R (Pseudo R 2 ) Wald χ p > χ No. of observations * No. of participants PRWL = WTAT χ 2 = 2.71 p = χ 2 = 4.06 p = χ 2 = 7.70 p = F = 1.43 p =

20 * 15 subjects did not provide gender information, 7 subjects did not complete the trait anxiety questionnaire properly, and 2 providing neither, thereby leading to a loss of 360 observations over 15 rounds. Standard errors in parentheses; p-values in square brackets. In Model 1 of Table 4, we see that the PRWL treatment dummy is negative while that for the WTAT treatment is positive. Neither of the coefficients are, however, significant at conventional levels. When we compare the PRWL and WTAT treatments, the Wald test shows that the PRWL treatment performs significantly better. From Model 1, we also see that learning occurs, with forecast errors improving over time. When we control for gender and trait anxiety in Model 2, we see that the average forecast errors are significantly lower in the PRWL treatment compared to the reference PR treatment. A Wald test also show that forecast errors are also smaller in the PRWL treatment compared to the WTAT treatment. What is also noticeable is the large positive coefficient for the gender dummy showing that, on average, women performed much worse than men across the board. In Model 3, when we allow for treatment-interacted time trends, we continue to see similar results. The coefficient on the PRWL dummy continues to be negative and the WTAT dummy remains insignificant, showing that performance in these treatments are, respectively, better than and no different to the PR treatment. Comparing the PRWL and WTAT treatment coefficients, the forecast errors continue to be smaller in the PRWL treatment. While the WTAT treatment does not appear to perform well compared to the other treatments, it is made up for by better learning. Of the treatment-interacted time trends, only the WTAT trend is significant and in the direction that indicates improved forecasts over time. We will come back to this shortly. As a robustness check, in Model 4 we run a quantile regression, which is robust to outlier values. However, quantile regressions do not accommodate the panel structure of our data, nor do they allow us to estimate time trends. The quantile regression in Model 4 corroborates the previous findings. On the basis of these results, we conclude that forecast errors in the PRWL treatment are smaller than in the PR and WTAT treatments, with no difference in performance between the latter two treatments. This provides support for each of our three hypotheses, where PRWL (Errors) < PR (Errors) WTAT (Errors). We find support for Piece Rate Equivalence, and we can infer from our results that it is driven by the motivating effects associated with 20

21 rank competition. When rank competition is controlled for in piece rates, tournaments do not perform as well. The regressions in Table 4 had pooled the single and dual cue treatments. We now repeat our regressions, but separating single and dual cue treatments. Table 5 consists of two random effects regressions for each of the single and dual cue tasks across rounds 6 to 20. Similar to Model 3 of Table 4, each of the regression models in Table 5 regress forecast errors against the treatment dummies (with PR as the reference treatment), gender (with women = 1), trait anxiety, as well as time trends for each treatment. In disaggregating the treatments by task difficulty, we are spreading the number of observations thin, effectively using half the number of observations compared to the previous pooled regressions in Table 4. In the single cue regression in Model 1 of Table 5, we see that although the PRWL treatment dummy is insignificant, it carries a negative sign similar to what we observed in the pooled regressions. The PRWL coefficient in the dual cue regression in Model 2 is also negative and insignificant, but is larger in magnitude compared to the corresponding single cue coefficient. While the signs on the PRWL coefficients for both single and dual cue regressions are consistent with that from the pooled regressions, the fact that neither are statistically significant suggests that the smaller number of observations plays a role. Consistent with what we found earlier for the pooled regressions, the WTAT treatment for both single and dual cue regressions remain insignificant. The Wald test shows that the PRWL treatment performs no differently from the WTAT treatment in the single cue regression in Model 1 of Table 5, while it performs better than the WTAT treatment in the dual cue regression of Model 2. The patterns of learning are also similar when we disaggregate our results by task difficulty. In both single and dual cue regressions in Table 5, the time trends estimated for each treatment show that it is negative and significant only in the WTAT treatment. 21

22 Table 5: Random effects regressions for forecast errors separated by Single and Dual Cue treatments Dependent variable = Forecast errors = Predicted value Actual Value Independent variables Model 1 Model 2 Single Cue Dual cue PRWL WTAT Female Trait anxiety PR X Round PRWL X Round WTAT X Round Constant (3.329) [0.449] (3.187) [0.939] (1.432) [0.359] (0.116) [0.270] (0.140) [0.260] (0.182) [0.836] (0.100) [0.050] (5.728) [0.002] (4.935) [0.145] (4.961) [0.205] (2.811) [0.000] (0.249) [0.312] (0.246) [0.606] (0.262) [0.833] (0.212) [0.020] (11.05) [0.264] R Wald χ p > χ No. of observations No. of participants χ PRWL = WTAT 2 = 0.92 p = Standard errors in parentheses; p-values in square brackets. χ 2 = 6.01 p =

23 4.1. Heterogeneous Ability In this section we disaggregate analyses by subjects ability. Since our MCPL task is difficult, there could be a large degree of heterogeneity in players ability, which would in turn adversely affect our statistical analyses. Furthermore, it is possible that rank and payoff competition have different effects for subjects of different ability, which are not reflected by our aggregated results. Recall that participants in each treatment play under piece rates in the first five rounds. Performance in those five rounds then provides us with a benchmark of participants ability. We use the median error in the first five rounds to categorise our participants into high or low performers according to how they performed during the first five rounds. We compute the median forecast error for each subject and those subjects whose median error exceeded the aggregate median during the first five rounds are treated as low performers while those whose median falls below the aggregate median are high performers. 9 In Table 6 we report results of random effects regressions similar to the ones we presented earlier, except we break this up by high and low performers as we have previously defined. As in Table 4, we pool data from the two tasks (single cue and dual cue) for the regressions. We report two regression specifications for each category of high and low performers. In the first specification, the regressors include two treatment dummies PRWL and WTAT (with PR as the reference category), controls for gender (Female = 1 for women, 0 for men) and trait anxiety, as well as a linear time trend denoted by Round. In the second specification, we include treatment interacted time trends instead of the aggregate trend. 9 The median error for all subjects across the first five rounds for the dual cue task is 21 while that for the first five rounds of the single cue task is 8. These serve as the ability thresholds which we use to categorise high and low performers. In the dual cue task a subject is classified as a high (low) performer if the median error for this subject during the first five rounds is less than or equal to (greater than) 21. In the single cue task a subject is classified as high (low) performer if the median error for this subject during the first five rounds is less than or equal to (greater than) 8. 23

24 Table 6: Pooled random effects regression for forecast errors broken up by high and low performers; columns (1) and 2 present results for high performers while columns (3) (4) presents results for low performers. Dependent variable = Forecast errors = Predicted value Actual Value Independent variables High performers Low performers Model 1 Model 2 Model 3 Model 4 PRWL WTAT Female Trait anxiety Round (2.114) [0.606] (2.831) [0.659] (2.110) [0.010] (0.130) [0.830] (0.072) [0.000] PR X Round --- PRWL X Round --- WTAT X Round --- Constant (5.416) [0.003] (3.423) [0.303] (4.107) [0.746] (2.111) [0.010] (0.130) [0.830] (0.124) [0.005] (0.108) [0.141] (0.138) [0.011] (5.810) [0.004] (4.242) [0.068] (4.993) [0.951] (3.181) [0.006] (0.215) [0.202] (0.155) [0.889] (9.304) [0.263] (5.743) [0.105] (5.336) [0.321] (3.183) [0.006] (0.216) [0.203] (0.272) [0.746] (0.319) [0.511] (0.200) [0.087] (10.06) [0.372] R Wald χ p > χ No. of observations No. of participants PRWL = WTAT χ 2 = 0.79 p = χ 2 = 1.89 p = χ 2 = 3.48 p = χ 2 = 5.71 p =

25 What stands out is that the performance of the high performers does not change across treatments. However, low performers perform better in the PRWL treatment than in the PR and WTAT treatments. In Model 3, we note that the dummy for the PRWL treatment is negative and significant (at 7%) indicating that the low performers committed smaller errors compared to the PR treatment. Pairwise Wald tests for the equality of coefficients suggest that the coefficient for the PRWL treatment is significantly smaller than that of the WTAT treatment. In Model 4, the coefficient of PRWL narrowly misses conventional levels of significance (p = 0.105). Once again, the pairwise Wald test suggests that compared to the WTAT treatment, performance is better in the PRWL treatment. In terms of learning, in Model 2 we observe significant learning for high performing PR and WTAT subjects. For low performers, from Model 4 we observe learning only in the WTAT treatment. To summarise: high performing subjects performed equally well across the different treatments, while low performing subjects performed particularly well in the PRWL treatment compared to the PR and WTAT treatments. It appears that our results are mainly borne out by low performers. As a robustness check, we can instead separate players by their ex-post record of winning. Since competition does not occur in the PR treatment, we will focus only on the PRWL and WTAT treatments. We define subjects to be winners if they have won 8 or more of the 15 post-intervention rounds which feature an element of rank or payoff competition, while losers have won 7 or fewer rounds. Table 7 present similar regressions to the ones before, where we regress forecast errors for winners and losers separately. In Models 1 and 3, we regress forecast errors against the WTAT treatment dummy (with PRWL as the reference category), the gender and trait anxiety of players as controls, as well as a time trend. In Models 2 and 4, we replace the trend with time trends specific to the PRWL and WTAT treatments. 25