Inattentive Inference

Transcription

1 Inattentive Inference Thomas Graeber October 15, 218 Abstract Most information structures have multiple unobserved causes. Learning about a specific cause requires taking into account other causes in the signal structure. For example, a principal who infers an agent s effort from his performance needs to factor in other inputs such as luck. A failure to consider alternatives generates misattribution to the cause of interest. Using a series of laboratory and online experiments, this paper demonstrates that people are inattentive to alternative causes, leading to excessively sensitive and overprecise beliefs. This neglect is driven by people s unawareness about the need to take into account other sources of randomness in a signal structure. Higher effort is typically insufficient to overcome this unawareness, but contextual cues that shift attention to the neglected part of the problem reduce the bias. The evidence from more than twenty treatments is consistent with a model in which automatic, simplified representations of an inference problem form the basis for how new information is processed. JEL Classification: C91, D1, D83, D84 Keywords: Belief, Attention, Bounded rationality, Learning, Unawareness This work has benefited from generous comments by Pedro Bordalo, Katie Coffmann, Stefano DellaVigna, Xavier Gabaix, Benjamin Enke, Christopher Roth, Kilian Russ, Frederik Schwerter, Justus Winkelmann, Florian Zimmermann as well as audiences at Bonn, Harvard, the ECBE 218 conference, SITE Summer Workshop 218 in Experimental Economics and CRC TR 224 of the German Research Foundation. Most of all, I thank Armin Falk and Andrei Shleifer for their guidance. Financial support from the Bonn Graduate School of Economics and the Institute on Behavior and Inequality (briq) is gratefully acknowledged. Graeber: Harvard University, Department of Economics, graeber@fas.harvard.edu.

2 1 Introduction Most outcomes have multiple unobserved causes. Performance on a test, for example, might depend on effort and luck. Learning about a specific cause, e.g., effort, requires accounting for other causes in the information structure, i.e., luck. This paper studies how people learn about a state of the world from signals that also depend on other unobserved states. The crux is that a failure to account for alternative causes creates misattribution to the cause of interest. For example, a person who does not factor in the role of luck would over-attribute a bad test performance to low effort. Empirical evidence consistent with this hypothesis comes from CEOs and politicians who are partly rewarded for luck because performance evaluations and voter support often fail to condition on external conditions such as the business climate (Bertrand and Mullainathan, 21; Wolfers et al., 22). Similarly, in situations involving prosocial behavior and reciprocity, people overstate the role of intentions relative to contextual factors and chance in generating adverse outcomes (Charness and Levine, 27; Gurdal et al., 213). Using a series of laboratory and online experiments, this paper provides two main findings. First, people neglect alternative causes in signal structures when they want to learn about a specific cause. Such misattribution, labeled feature neglect, leads to predictable patterns of overshooting and overprecision in beliefs. Second, I investigate the underlying cognitive mechanisms and show that feature neglect is not driven by the inability to understand and solve the problem, nor by excessive complexity or a lack of effort. Instead, subjects fail to detect the need to account for alternative causes to begin with. This unawareness prevents them from forming Bayesian beliefs even when they exert substantial effort. Contextual cues that draw attention to other inputs in the signal structure are required to overcome the neglect. In the experiment, subjects guessed an unknown state of the world and were paid for accuracy. Before indicating their guess, they received a piece of information that depended both on that state and another unobserved state. Subjects knew the datagenerating process and the signal structure, eliminating all structural uncertainty about the information environment. Specifically, two numbers X and Y were drawn from known distributions. They had to guess X, but not Y. In a typical task, X was drawn from the simple discretized uniform distribution {3, 4, 5, 6, 7} and Y was drawn at random from {1, 2, 3, 4, 5, 6, 7, 8, 9}. Subjects then observed a signal about the average of the drawn numbers, S = X+Y 2 = 6. Crucially, inference 1

3 from S about X requires accounting for the variation in Y. In the context of the motivating example, a principal might want to infer the agent s unobservable effort (X) from his observed performance (S), which is the average of effort X and luck Y. Failing to properly account for the distribution Y gives rise to mis-attribution of the signal to X. Here, a person might mistakenly conclude that X = 6 with certainty. The key idea of the experimental design is a between-subjects comparison of the beliefs in this treatment to those stated in a control condition in which subjects did not need to account for alternative causes, but still faced the identical information environment. This was achieved by a variation of the structure of prediction incentives: in the control condition, subjects guessed both X and Y, and were paid for the accuracy of either one of their guesses. In each of five tasks, subjects received either the average or the sum of the drawn numbers as signal. Forming optimal beliefs was particularly simple in this setting. Subjects had to come up with the combinations of X and Y that could generate the signal, all of which were equally probable. I elicited the distribution of each subjective belief rather than a point belief. The baseline experiments provide three main findings. First, beliefs in the main treatment exhibited feature neglect, indicating that subjects failed to account for Y on average. In most Second, there was a large and statistically significant treatment difference. In fact, the median belief in the control condition, where subjects guessed both X and Y, was indistinguishable from the Bayesian posterior. This demonstrates that the experimental setting was not too complex and subjects were in principle able to solve the problem correctly. Third, the empirical distribution reveals that the vast majority of beliefs either exactly corresponded to feature neglect or was close to the Bayesian benchmark. For a comprehensive account of feature neglect, I designed additional experiments. First, I document that the phenomenon is robust to variations of the data-generating process and signal structure. I rule out alternative explanations, including a treatment difference in the elicitation procedure or a simple signal anchoring heuristic. Second, I characterize the functional form of feature neglect. Feature neglect does not generally comply with an (implicit) simplification of the distribution of Y, such as replacing it by its mean or shrinking its variance, nor with overweighting the signal or underweighting the base rate. Instead, the bias is best described as a strong form of unawareness about the role of Y, as if X were the only source of randomness in the signal structure, S = f (X). Third, to gain a 2

4 better understanding of belief heterogeneity and updating rules, I employ a simplified, more natural version of the baseline design. The main finding replicates in a large, heterogeneous online sample from Amazon Mechanical Turk. Next to feature neglect and Bayesian updating, I document the existence of a smaller, third mode that corresponds to non-updating, a typical finding in belief formation studies. The resulting, pronounced tri-modality of the belief distribution points to the presence of precise updating rules. The second part of the paper investigates the origins of feature neglect. Understanding the underlying cognitive mechanisms holds the promise of uncovering common primitives of different updating errors (Fudenberg, 26), of increasing the ability to predict its occurrence, and of identifying strategies and policies to debias people (Handel and Schwartzstein, 218). A simple conceptual framework organizes the empirical analysis. The framework posits that belief updating relies on people s mental representation of a situation, i.e., how they think about a problem. 1 Representations are people s internal models of the external world and serve as the basis for subsequent computations, reasoning, and action. Relatedly, recent theoretical and empirical work in economics explores the consequences of misspecified models of the world. 2 The framework builds on two key insights about the nature of default representations. Default representations (i) are characterized as emerging automatically and effortlessly, are geared to the immediate task demands and taking action, tend to be simple, low-complexity models, and (ii) they are not easily overturned once they have formed. It usually takes a cue to trigger a different representation. In the framework, feature neglect results from the structure of default representations, which lead people to neglect alternative causes in belief updating problems. Using a set of new experimental designs, I empirically investigate the mechanisms underlying feature neglect in light of this framework. In particular, I test the plausibility and distinct implications of a representation-based model of updating as opposed to standard theories that rely on variations of weighing the expected benefits against the cognitive costs of inattention. If feature neglect is driven by a flawed representation of the problem that omits Y, 1 Representations are a fundamental theme in cognitive science, see, e.g., Clark (213), Fodor and Pylyshyn (1988), and Newell and Simon (1972). 2 See, e.g., Barron et al. (218), Bohren (216), Bohren and D. Hauser (217), Enke (217), Enke and Zimmermann (217), Esponda and Pouzo (216), Eyster and Rabin (214), Hanna et al. (214), and Schwartzstein (214). 3

5 its prevalence should respond to cues that shift attention to Y, but not necessarily to consciously exerted effort. This is because representations are characterized by their automaticity and are not subject to direct control, unlike the computations performed on those representations. Supporting that prediction, I found that increasing the incentives tenfold (in the laboratory) or fivefold (online) substantially increased effort as measured by response times, but did significantly reduce feature neglect. This suggests that people were unaware of the need to account for Y to begin with. Intriguingly, the share of non-updates fell dramatically in the presence of high incentives, suggesting that the decision to form an update or not reflects cost-benefit considerations. Next, I designed two treatments to test the effect of cues on feature neglect. The first treatment added the following verbal hint to each elicitation screen: Also think about the role of Y. This intervention virtually eliminated feature neglect, in line with the idea that subjects initially fail to detect the need to account for Y. The second treatment enforced a 3-second deliberation time on the elicitation screen before the input fields were activated. This presumably nudged subjects to rethink their solution strategy for the problem. The deliberation time indeed substantially reduced feature neglect, by roughly half as much as the hint. If the solution process involves two steps, i.e., the automatic formation of a representation followed by computations within that representation, then higher stakes induced subjects to exert more effort on the second step while the representation remained unchanged. A hint or enforced deliberation targeting the first step were necessary to broaden the default representation. Further analyses substantiate and extend the above evidence on mechanisms. They provide four main insights. First, I examined whether the presence of feature neglect systematically responds to the characteristics of the information environment. In models with rational allocation of attention, the use of non-bayesian updating rules is driven by how costly the associated distortions are expected to be (Caplin and Dean, 215; Gabaix, 214). Default representations, by contrast, are insensitive to slight variations in the information environment as long as the basic structure of the problem is maintained. The framework thus predicts that the propensity to commit feature neglect is not highly responsive to the anticipated accuracy of feature neglect. By contrast, an information environment should lead to reduced feature neglect if the observed signal realizations are subjectively implausible under the default representation. For example, for a person who treats the signal as being generated by S = X, an observed signal s outside of the support of X is 4

6 subjectively a zero-likelihood event and acts as a cue that triggers reconsideration. I tested these predictions in two experiments that varied the variance and mean of Y relative to that of X, changing the amount of noise and bias introduced into the signal structure by Y, respectively. The results uniformly support the framework s prediction of an ex-post responsiveness to the information structure, i.e., driven by the plausibility of the observed signal value under the default representation, but no ex-ante responsiveness based on the expected distortion associated with feature neglect. Second, I investigate people s awareness about feature neglect directly by measuring confidence in their beliefs using incentivized willingness-to-pay statements. If subjects who commit feature neglect were at least minimally aware about the need to account for Y, they should be less confident in the resulting estimate. I find that feature neglect is associated with similar confidence levels as Bayesian updating, suggesting that subjects are unaware about the neglect. There is, however, considerably lower confidence in non-updates, supporting once again the notion that whether an update is formed at all is a deliberate choice. Third, I study the implications of representation-based updating for the persistence of feature neglect. In the framework, the extent of learning from feedback might be limited, because subjects only remember the consciously executed computations, but are unaware of their underlying problem representation. In three treatments I provide feedback on the draw in the previous round and vary the amount of computational steps required to arrive at a solution, holding fixed the problem s complexity. I document that learning from feedback is indeed targeted at the consciously executed computations in a solution process. A larger number of required computations therefore interferes with overcoming feature neglect. Fourth, I examine how different updating rules are linked to individual-level characteristics, specifically subjects cognitive skills. While research in economics typically equates cognitive skills with intelligence, the framework makes more precise predictions that leverage insights from cognitive science. Next to fluid intelligence, defined as algorithmic cognitive capacity or the computational power that is in principle available, a person s cognitive performance has been shown to depend on his disposition for reflective thinking, i.e., the tendency to mentally construe problems in a comprehensive way (Stanovich, 29; Stanovich and West, 1997, 28). The latter affects an individual s capacity for override detection. In support of the framework, the propensity to commit feature neglect was strongly correlated with 5

7 a measure of reflective thinking at the individual level (Cacioppo et al., 1996), but unrelated to a standard intelligence measure (Raven et al., 23). 2 Baseline Evidence for Feature Neglect 2.1 Design A causal investigation of belief formation given compound signal structures requires (i) a fully controlled and transparent data-generating process and information structure that is known to subjects, (ii) an experimental manipulation of whether a signal is compound or specific, (iii) limited complexity to minimize confusion, (iv) a clear prediction for the posterior under feature neglect, and (v) an incentive-compatible procedure to extract beliefs. Below I present a tightly controlled design that meets these criteria. The laboratory provides maximum control over the information structure, the choice setting and the sample characteristics. The baseline laboratory experiment is designed as a causal test for the existence of feature neglect under tightly controlled conditions. To address concerns about differential complexity when using different signal structures across conditions, I create an environment that allows me to vary whether a signal is compound or specific without changing the signal structure or data-generating process. The least complex setting of this kind features only two unobserved random numbers X and Y, generated by stochastic processes known to subjects. To simplify further, these numbers are independently drawn from two discrete uniform distributions with small sample spaces. Subjects then receive a signal S on the two unknown draws that is easy to understand: depending on the task, they see either the sum or the average of the two numbers. The signal structure maps two inputs, i.e., the realizations of random variables X and Y, to a one-dimensional output, i.e., the observed signal s. The experiment induces exogenous variation in whether the agent s optimal action depends on only one or both of the inputs in the signal structure. Accordingly, there are two experimental conditions in the baseline design: In Narrow, subjects are paid to guess only X, while in Broad, subjects are paid to guess both X and Y. The induced prior, the signal structure, and the Bayesian posterior are identical in Narrow and Broad. Moreover, by randomly paying for only one of the guesses in Broad (about X or Y), the incentive size is kept constant. This 6

8 implies that a Bayesian agent forms identical beliefs in both conditions. Table 1: Overview of baseline tasks Sample space X Sample space Y Signal structure Signal value 3, 4, 5, 6, 7 1, 2, 3, 4, 5, 6, 7, 8, 9 (X + Y) , 19, 2, 21, 22 18, 19, 2, 21, 22 (X + Y) , 14, 15, 16, 17-25, -15,-5,, 5, 15, 25 X + Y 165 8, 9, 1, 11, 12-3, -2, -1,, 1, 2, 3 X + Y 8 23, 24, 25, 26, 27 21, 22, 23, 24, 25, 26, 27, 28, 29 (X + Y) 2 23 Notes: This table provides an overview of the five baseline belief tasks in the laboratory study. The distributions of X and Y as well as the signal structure were identical in both treatment conditions. X and Y were independently drawn from two discrete uniform distribution, i.e., every indicated outcome was initially equally likely. Subjects solved the five updating problems of Table 1 in random order without receiving feedback in between. For example, in the first task of of Table 1, X was one of five numbers, 3, 4, 5, 6 or 7 with equal probability, while Y was independently drawn as a multiple of 1 between 1 and 9. Subjects learned that the average of X and Y was 6 and then stated their belief as described in detail in Section To solve the problem, subjects would need to identify all (X, Y) combinations with an average of 6, that is (3, 9), (4, 8), (5, 7), (6, 6), (7, 5). Both numbers being drawn uniformly and independently, it is intuitive that each of these outcomes is equally probable. The elicitation procedure extracted the maximum amount of information about subjective beliefs by having subjects indicate the full posterior distribution instead of point predictions. At the end, one of the tasks was randomly selected to be paid out based on the Binarized Scoring Rule with a prize of 1 euros (Hossain and Okui, 213). 4 Subjects receive extensive instructions and had to complete eight control questions that test their understanding of the instructions, the data-generating process and signal structure, as well as the elicitation protocol. In two unpaid practice tasks subjects were trained to indicate a given belief in a way that would maximize their payoff. This training stage was identical in both treatments. A notable feature of this design is that unlike previous empirical studies of belief formation, the present experiment holds the information environment fixed across 3 Note that in the baseline experiment, the numbers were drawn jointly for all subjects by the computer, hence all participants saw the identical signal value. 4 The scoring rule proposed by Hossain and Okui (213) elicits truthful beliefs even if subjects are risk averse or do not follow the expected utility hypothesis. 7

9 conditions, mitigating concerns about differential complexity (Caplin et al., 211; Dean and Neligh, 217; Enke, 217; Enke and Zimmermann, 217; Khaw et al., 217). Beyond the tasks in Table 1, a number of different task specifications and additional treatment variations address robustness of the baseline results and examine the nature of updating rules (see Sections 2.6 and 3.2). 2.2 Baseline Prediction: Feature Neglect in Treatment Narrow The Bayesian posterior belief about X given the signal is characterized by a discrete probability distribution P(X S) = P(S X) P(X). This normative benchmark (i) applies P(S) independent of the decision maker s incentive structure and (ii) always depends on Y through S. Accordingly, the treatment manipulation is inconsequential under Bayesian updating. Consider instead a person who selectively attends to the variables he perceives as being most important given his incentive scheme. In condition Broad, X and Y are equally important for the decision maker s payoff. In condition Narrow, however, the realization of Y neither changes the decision maker s optimal action, i.e., her optimal prediction about X, nor does it affect her payoff given an action. She might be inattentive to and (partially) neglect Y in the updating problem, which I call feature neglect. Note that if inattention leads to a neglect of Y, it is a priori unclear which form this neglect takes. For example, the decision maker might underestimate the variance of Y, replace Y with a default value, be in some way unaware about its role in generating the signal, or apply a specific non-bayesian updating rule in the belief formation process. Different hypotheses about the form of the neglect require different auxiliary assumptions, such as a default value. The motivation for the design of the baseline tasks was to facilitate the detection of different plausible ways of neglect by imposing minimal assumptions on its parametric structure. 5 In all baseline tasks of Table 1, the information structure is an unbiased estimator of the mean of X. Either subjects receive the average of the drawn numbers and the prior distributions of X and Y have an identical mean, or they see the sum of the drawn numbers and Y has a mean of zero. This provides a natural way of how Y can be neglected, namely by interpreting the information as a specific, non-confounded signal about X. Specifically, I assume that feature neglect can be characterized by I investigate the precise form of feature neglect in additional experimental variations, see Section 8

10 the agent updating based on a subjective signal structure S i = X. The agent builds a posterior P(X S i = s) instead of P(X S = s), based on a simplified subjective signal structure. 6,7 Anticipating the findings in Section 3.2, note that feature neglect indeed turned out to be best characterized by the (implicit) use of a modified subjective signal structure. Prediction 1. Beliefs formed in Narrow and Broad significantly differ. Subjects in condition Narrow display feature neglect. 2.3 Procedures Subjects in condition Broad guessed the joint distribution of X and Y and were randomly paid for their accuracy in guessing either of these. The decision screen is displayed in Appendix Figure A24. Subjects in condition Narrow only guess the marginal distribution of X (Appendix Figure A21). 8 To reduce potential experimenter demand, the design unobtrusively obfuscates the experimental objective. Subjects received their signal in encrypted form and had to decipher it using a simple twostep decoding protocol. 9 No subject had trouble implementing the protocol. Each belief elicitation (excluding the deciphering stage) was subject to a time limit of five minutes. The findings are robust to removing both the deciphering and the time limits (see Section 2.6). The belief updating problems were followed by a questionnaire. To shed light on correlates of subject-level heterogeneity in belief 6 Feature neglect in condition Narrow is observationally equivalent to taking the observed information at face value for the unobserved X. I demonstrate that belief formation here is not driven by anchoring on the signal value. First, anchoring cannot explain a treatment effect because the signal is identical across treatments. Anchoring should similarly affect beliefs in condition Broad. Second, the additional treatment variation Computation explicitly rules out anchoring effects, see Section Given S i, it is possible that the agent observes signals to which he assigned zero probability. I will show in Section 3.2 that, empirically, subjects in this case tend to update as if they observed S i with some error ε, and form P(X S i = s + ε) such that P(S i = s + ε) >. The higher the perceptional error ε required to rationalize the signal, however, the more likely it is that people update in Bayesian fashion instead. 8 While this is the most natural and my preferred design to test for feature neglect, note that there is a treatment difference in what is elicited, namely X and Y versus only X. Additional treatment variations harmonize the elicitation protocol, i.e., subjects with Narrow incentives predict both X and Y, and subjects with Broad incentives predict first the marginal of X, and then the marginal of Y on a separate subsequent page. All main findings persist. See Section Concretely, subjects saw a sequence of letters. First, each letter had to be translated into a digit based on a displayed table. Then the number 2 had to be added to the result. Subjects were familiarized with the deciphering process in the practice stage. See also the instructions reproduced in Appendix F. 9

11 formation, I measured performance on a incentivized test of cognitive capacity (1 Raven matrices,.2 euros per correct answer) and elicited a measure of risk preferences (Falk et al., 216) as well as two personality questionnaires, the Big 5 inventory (Rammstedt and John, 25) and the Interpersonal Reactivity Index (Paulus, 29). 144 student subjects (72 in each treatment) participated in six sessions of the baseline experiment run at the University of Bonn s BonnEconLab in July 217. Treatment status was randomized within-session. I invited subjects using hroot (Bock et al., 214) and implemented the study in otree (Chen et al., 216). Mean earnings amounted to 11.4 euros including 5 euros show-up fee for an average session duration of 57 minutes. 2.4 Baseline Results Result 1. Beliefs in Narrow significantly differ from Broad in line with feature neglect. Figure 1 displays the distribution of raw beliefs in all five baseline tasks. It shows the sample distribution of subjective belief distributions for both conditions, as well as the Bayesian benchmark and the value of the observed information. The average subject in Broad forms beliefs that are closely aligned with the Bayesian posterior. In Narrow, by contrast, subjects on average assign too much probability mass to outcomes close to the signal value, as implied by inattention to Y. Three implications of these results are that (i) there is no systematic misunderstanding of the experimental setup, since most subjects in Broad successfully arrive at Bayesian beliefs, (ii) in Narrow, beliefs overshoot in the direction of the information value and (iii) are overprecise relative to Bayesian and Broad beliefs. Task (3) in Figure 1 exemplifies the role of overprecision. Since the signal realization coincides with the mean of the Bayesian posterior distribution, subjects in Narrow form beliefs featuring the correct expected value of X. They are, however, far too faithful that this expected value of X equals the actual draw. This observation would not be possible if only point predictions were elicited. Table 2 provides an overview of summary statistics and non-parametric tests by task. Median beliefs in Narrow (column 3) and Broad (column 4) closely correspond to the observed information (column 1) and Bayesian benchmark (column 2), respectively. Column 7 shows that belief distribution means and belief distribution 1

12 variances are significantly different between treatments at the.1% level (M-W U tests). 1 Table 2: Beliefs about X in baseline tasks Observed information Bayesian posterior distribution Narrow N=72 Subjective posterior distribution Broad N=72 Sign test of median Narrow vs. Bayesian Broad vs. Bayesian M-W U test Narrow vs. Broad (1) (2) (3) (4) (5) (6) (7) distribution mean (distribution variance) median of distribution means (median of distribution variances) p-value: distribution of means (p-value: distribution of variances) < <.1 (2) () (2) (<.1) (.11) (<.1) <.1 <.1 <.1 (71.7) () (67) (<.1) (<.1) (<.1) (2) () (2) (<.1) (.4) (<.1) <.1.58 <.1 (125) () (125) (<.1) (.18) (<.1) <.1 1. <.1 (125) (25) (125) (<.1) (.18) (<.1) Notes: This table displays beliefs in Narrow and Broad for each one of the five baseline tasks. An elicited belief corresponds to a full distribution, which is described here by its mean and variance. I show medians of subjective distribution means and variances in each condition and compare these to the mean and variance of the Bayesian posterior distribution. Column (7) shows treatment comparisons for the distributions of distribution means and variances. The task order was randomized. 1 This holds for all tasks except the distribution means in task (3), in which the observed information coincides with the Bayesian posterior mean. 11

13 Distribution of beliefs (1) (2) (3) Distribution of beliefs (4) (5) Condition Narrow Condition Broad Bayesian posterior Figure 1: Distribution of elicited belief distributions about X in each one of five baseline tasks. N=72 for each condition in each task. The horizontal axis shows possible outcomes of X. The Bayesian posterior belief is provided for reference. The observed signal is indicated by the vertical dashed line. In all five tasks, X and Y follow independent discrete uniform distributions that were shown to subjects. The task order was randomized at the subject level. The distributions and signal structure of all tasks are provided in Table 1. Subjects observed the mean of the drawn numbers in tasks (1), (2) and (3), and they saw the sum in (4) and (5).

14 Feature neglect comes at a sizeable cost for subjects. The average expected payoff for the beliefs stated in the baseline tasks was 53% higher in Broad than in Narrow (5.86 versus 3.82 euros, p <.1, M-W U test) Typical Beliefs and Updating Rules The measures of central tendency analyzed in Section 2.4 may obfuscate the presence of specific updating rules. Next I examine what are typical beliefs in each condition. For this purpose I characterize each stated belief by how close it is to the Bayesian posterior relative to the posterior under feature neglect, recognizing that an observation corresponds to a full distribution rather than a single value. To obtain a measure of distance between distributions, I first calculate the Hellinger distances (Hellinger, 199) between the stated posterior P and the Bayesian posterior P B : 12 H B = 1 k ( 2 P(X i ) i=1 P B (X i )) 2 (1) Given an analogous distance to the inattentive posterior distribution, H FN, 13 I define an inattention score θ that captures the distance of the subjective belief distribution to the Bayesian distribution, relative to the sum of the distances of the subjective distribution to the inattentive and the Bayesian posterior: θ = H B H B + H N (3) A Bayesian belief corresponds to θ = and feature neglect to θ = 1. The parameter θ can be computed for every stated belief, independent of the updating task, so I 11 Actual earnings for the baseline tasks also significantly differed across groups (means of 4.56 in Narrow and 2.22 euros in Broad, p =.5, M-W U test), but these further depended on randomness induced by the binarized scoring rule as well as an additional choice by subjects that affected their payoff (see Section 5.3). 12 The Hellinger distance is a bounded metric frequently used to characterize the similarity between two probability distributions (Bandyopadhyay et al., 216). It is suited for the present purpose as it is a proper metric, unlike, e.g., the Kullback-Leibler divergence, which does not satisfy symmetry. 13 H FN is calculated as: H NN = 1 k ( 2 i=1 2 P(X i ) P FN (X i )) (2) 13

15 proceed with a joint analysis of the data pooled together from all tasks. Figure 2 is a histogram of empirical inattention parameters, split by treatment condition. More than 7% of beliefs are roughly Bayesian in Broad, whereas little short of 2% are in Narrow. Instead, about 6% in Narrow lie within a small window around full feature neglect, with the remaining 2% located in between the two poles. This figure indicates that the vast majority of stated beliefs are either fully sophisticated or fully inattentive to Y. Based on this measure of inattention, which does not systematically characterize other possible benchmarks for stated beliefs, the distribution of beliefs is markedly bi-modal..8.6 Fraction of beliefs Estimated inattention parameter Condition Narrow Condition Broad Figure 2: Inattention to Y in baseline tasks. N=1135. Indicated are treatment-specific binned histograms for the implied inattention parameters from all beliefs elicited in the five baseline tasks. Inattention is calculated as θ = H B H B +H N, where H B and H N denote the Hellinger distance of the subjective distribution to the Bayesian posterior and the inattentive posterior distribution, respectively. A parameter of θ = is consistent with Bayesian updating. θ = 1 is a fully inattentive belief. Before moving on to investigate the heterogeneity of beliefs as well as the form of feature neglect in Section 3, I consider potential confounds in the following. 14

16 2.6 Robustness Checks The baseline study documents feature neglect in a specific configuration of the information environment and experimental setup. In additional experiments I address potential confounds and examine the robustness the findings. These extensions include (i) additional tasks introducing various departures from the simple discrete uniform case, (ii) a direct test of a signal anchoring heuristic, (iii) two treatments that exactly align the elicitation procedure across conditions, and (iv) a simplified version that removes the deciphering stage and time limits. The following provides a brief discussion of these analyses, with all details delegated to the appendix. Four additional tasks were presented in random order after the baseline tasks. 14 First, moving toward a continuous data structure increased the complexity of the inference problem and pushed the median subject in Broad away from the Bayesian benchmark. Second, normal instead of uniform data appears to have had a similar added-complexity effect on subjects in Broad. Third, a signal value outside of the range of X woke subjects up in Narrow to some extent in that it increased the share of Bayesian beliefs. However, a large share of subjects simply jumped to closest value in the support of X, a pattern that occurs more generally as will be shown in Section 3.2. Fourth, if X and Y were correlated instead of independent, the median subject in Narrow displayed lower inattention. One explanation is that subjects accommodated the additional incentive to attend to Y that is induced by the correlation. At the same time, the presentation format of the distributions was varied in this task to illustrate the correlation, which plausibly affected subjects perception of the problem and so this task allows no definite conclusion. Most importantly, highly significant treatment effects persist in all four tasks (p <.1, M-W U tests). Treatment Computation directly tested whether feature neglect is driven by a simple face value heuristic, whereby inattentive subjects anchor their guess of X on the observed information value. 15 If this were the case, then the observed inattentiveness to Y might not be the specific neglect of Y, but only an instance of a specific simplification strategy. Treatment Computation is identical to Narrow, but inserts a simple algebraic computation into the signal structure, such that it remains equally plausible to anchor on the observed signal value. For example, instead of S = X+Y 2, subjects received the modified signal S X+Y 2 (2 1) + 3. I find almost no evidence 14 See Appendix C.1 for the robustness task specifications and detailed results. 15 Even then, anchoring cannot explain the treatment effect without further assumptions. 15

17 for anchoring on the observed signal. Instead, subjects were able and willing to invert the computations, but then still did not account for Y. 16 Computed inattention scores for beliefs in Computation are indistinguishable from Narrow (p =.37, M-W U test), and significantly different from Broad (p <.1). This suggests that the baseline finding reflects a specific error in probabilistic reasoning rather than mere anchoring on the signal value. Another block of treatments addresses the sensitivity of the baseline findings to specific experimental procedures. A central insight of the experiment is that attention can be directed using incentives, which in turn affects belief formation. The experiment, however, varied the elicitation procedure along with the incentive structure: subjects in Narrow only stated a belief about X, whereas subjects in Broad guessed both X and Y. 17 To better understand what portion of the treatment effect can be explained by the difference in the elicitation procedures, two additional treatments were designed to obtain a full 2 (incentives Narrow/Broad) 2 (elicited: only X / X and Y) between-subjects factorial design. I find that given an incentive structure, i.e., Narrow or Broad, harmonizing the elicitation protocol reduces the treatment effect by roughly one third, while all differences in estimated inattention scores remain highly significant (see Appendix C.3). Put differently, the major part of the treatment effect appears to be driven by prediction incentives rather than the elicitation procedure. Finally, drastically simplifying condition Narrow by removing the deciphering stage as well as all time limits induces a reduction in inattention (p <.1), but the treatment effect persists in a conservative comparison against the baseline condition Broad which includes deciphering and time limits (p <.1). The robustness exercises substantiate the baseline findings about the prevalence and distinctness of noise neglect. All details are relegated to Appendix C. 16 Further treatment details, figures and results are relegated to Appendix C This was a deliberate design choice. The natural setup to study this question appears to be the one in which a person only predicts the states with non-zero prediction incentives, since making a prediction in itself can provide a non-monetary incentive to pay attention. At the same time, the available information set was held exactly constant across treatments, so that subjects in Narrow did not have to memorize the distribution of Y. 16

18 3 Heterogeneity and the Functional Form of Feature Neglect The baseline findings pose a challenge for existing models of belief formation. A more detailed characterization of the updating patterns is necessary to trace out the empirical and theoretical implications of feature neglect. This section attempts to deepen the understanding of updating from compound signals by investigating the heterogeneity of updating rules more closely as well as by pinning down the functional form of feature neglect. 3.1 Heterogeneity and Replication in a Large, Heterogeneous Sample The laboratory experiments provide evidence for feature neglect in a controlled environment. The design is tailored to a causal treatment comparison, but may be of limited informativeness for the distribution of updating types in practice due to the following reason: the experimental procedure puts strong emphasis on the signal, with, e.g., a signal deciphering stage that makes it practically impossible to ignore it later on. 18 Moreover, subjects have to indicate a full posterior while the prior is uniform, such that stating the full prior distribution is comparably effortful. Furthermore, the baseline lab design does not allow to distinguish signal neglect from Bayesian updating in some tasks. I complement the laboratory evidence on feature neglect with online experiments that address the above issues and serve two objectives. First, the online study uses a simpler, generalized version of the laboratory design that is less centered on the causal identification of feature neglect (by means of treatment comparisons) but allows for a precise identification of heterogeneous updating patterns under more natural conditions. It also serves to test the robustness and generalizability of the laboratory results and allows precise and inexpensive replication by other researchers. Online experiments typically provide access to a larger, more heterogeneous population observed under less controlled choice conditions. Second, I study the nature of updating rules in a large variety of information structures and explore predictors of heterogeneity in belief formation. The online setting allows to 18 In many studies of belief updating, however, a substantial minority of non-updates is observed. 17

19 run multiple treatment variations with a large number of participants, which would be infeasible in the laboratory Design There are three main modifications relative to the laboratory experiment. First, subjects do not have to indicate a full posterior distribution but are incentivized to state the mean of their posterior belief, substantially simplifying the procedure. Second, X and Y are not discrete with sample space size below ten, but follow (discretized) continuous distributions with a much larger sample space. The baseline tasks are displayed in Table 3. Third, there is no deciphering stage preceding the belief elicitation. 19 Prediction 2. The main finding of feature neglect replicates in the online experiment Procedures I conducted incentivized experiments on Amazon Mechanical Turk (MTurk), an online labor marketplace frequently used by researchers. A recent study suggests that MTurk workers are more attentive to instructions than college students (D. J. Hauser and Schwarz, 216). Participants in my online experiments had to live in the U.S. and be of legal age, have an overall approval rating of more than 95 percent, and have completed more than 1 tasks on MTurk. Workers were paid.5 dollars for participation and could earn up to 5 dollars for their performance on the guessing task. They played five rounds in randomized order. An example of the urn-based representation of distributions is reproduced in Appendix Figure A25. One round was randomly chosen to be paid, and the payoff was determined based on a quadratic scoring rule. 2 In the online experiments, all subjects were paid to predict X only, analogous to condition Narrow in the laboratory experiments Moreover, in the online study X and Y were drawn by the computer at the individual level instead of jointly for all subjects. 2 The monetary payoff (in US dollars) was determined by the following rule: max {, 3.1 (guess of X draw of X) 2} 18

20 subjects participated in the online baseline experiments for an average payment of 2.7 dollars. Completion of the study took 13 minutes on average. It was implemented using otree (Chen et al., 216). Table 3: Online baseline tasks X Y I N (1, 1) N (, 1) X + Y N (1, 1) N (, 4) X + Y N (1, 4) N (, 1) X + Y U[75, 76,..., 125] U[ 25, 24,..., 25] X + Y X+Y U[75, 76,..., 125] U[9, 91,..., 11] 2 Notes: This table provides an overview of the five baseline belief tasks in the online experiment. For all normally distributed variables, the support was discretized to integers, truncated at µ 5 and µ + 5 and then the distributions were scaled such that the they have unit probability mass Baseline Results: Online Study Result 2. The distribution of subjective beliefs was trimodal. The three modes correspond to in order of frequency feature neglect, Bayesian updating and signal neglect. Figure 3 shows all stated beliefs together with the signal values observed in the five baseline tasks. It further highlights which stated beliefs would correspond to feature neglect, signal neglect and Bayesian updating. There is evidence for each of those three updating rules. In each task at least 6% of stated beliefs were exactly in line with these three updating modes. Among the three modes, Bayesian updating and feature neglect were observed with roughly similar frequency, while signal neglect occurred to a lesser extent. The Bayesian benchmark changes across tasks, and subjects clearly responded to this change. To illustrate the degree to which beliefs were clustered on these three updating modes, Figure 4 plots kernel density estimates for the task in the upper left corner of Figure 3. In this task, X N (1, 1), Y N (, 1), and S = X + Y. The stated belief that corresponds to a Bayesian posterior in this case was 1 + λ (s 1) σ2 X where λ = = 1 σx 2 +σ2 2. Intuitively, since X and Y have equal variance, a normatively Y optimal guess of X would attribute half of s s deviation from the expected value of 1 to X. Signal neglect, in turn, would correspond to a belief equal to the prior of 19

21 X = N(1,1) Y = N(,1) S = X+Y X = N(1,1) Y = N(,4) S = X+Y X = N(1,4) Y = N(,1) S = X+Y Guess about X X = U{75, 76,..., 125} Y = U{ 25, 25,..., 25} S = X+Y X = U{75, 76,..., 125} Y = U{9, 91,..., 11} S = (X+Y)/ Signal Bayesian posterior Noise neglect Signal neglect Figure 3: Beliefs in baseline tasks of online experiments. N=131 in each task. Each dot corresponds to one stated belief. The three red lines indicate the Bayesian benchmark, noise neglect, and information neglect. X, E[X] = 1. This is equivalent to assigning none of the deviation of s from its expected value to X. In fact, with m denoting a subject s stated guess, I can back out the empirical equivalent of λ, as ˆλ = m 1 s 1. In the case of information neglect with m = 1, ˆλ =. Finally, if people commit feature neglect they state m = s, leading to ˆλ = 1. Figure 4 provides three insights. First, most of the probability mass was centered on the three updating modes. Second, feature neglect was relatively most frequent in this task, and signal neglect least frequent. Third, as indicated by the rug plot on the right, people who neglected Y (ˆλ = 1) or the information (ˆλ = ) did so exactly. By contrast, people were more dispersed around the Bayesian benchmark ( ˆλ.5), presumably because it was harder to compute the Bayesian posterior exactly. In the experimental settings studied in this paper, beliefs are clearly too heterogeneous to be adequately described by a single representative updating rule. Average beliefs mask the underlying structure. At the same time, there is little randomness in 2

22 λ^ λ Figure 4: Kernel density plot for beliefs stated in a task where X N (1, 1), Y N (, 1), and S = X + Y. In this task, the Bayesian belief corresponds to 1 + λ (s 1) where λ = σ 2 X = 1 σx 2 +σ2 2. For each stated belief, the empirical counterpart of λ is calculated as ˆλ = m 1 s 1. The Y plot documents three distinct clusters at ˆλ = (signal neglect), ˆλ = 1 (feature neglect) and around ˆλ = 1 2 (Bayesian posterior.) Based on N=131. Epanechnikov kernel with bandwidth.7. stated beliefs. Instead, most beliefs accord to a discrete set of three updating modes. They align exactly with one of these modes, and there is virtually no mixing between the modes, i.e., people do not seem to choose combinations of updating rules. The main finding of substantial feature neglect in the laboratory replicated in the online study. In addition, I documented evidence for an additional updating mode, signal neglect or non-updating, which is in line with typical findings in studies on belief formation. The online study still built on the working assumption from Section 2.2 that the form of feature neglect corresponds to the (implicit) use of a subjective signal structure S i. Next I test and substantiate this assumption. 21

23 3.2 The Form of Feature Neglect To make the documented updating patterns tractable for models of belief formation requires understanding the form of the neglect. The term feature neglect, however, implies no immediate formal analogue. This is because there are, in principle, many ways in which Y can be neglected in the updating process. I characterize different possibilities by whether they correspond to the (implicit or explicit) use of (i) a modified signal structure S i, (ii) a modified distribution of Y, hy, or (iii) a non-bayesian updating rule. To illustrate this, assume that people updated as if the signal structure only depended on X, but not Y, i.e., S i = f (X). This has implications that can be distinguished from a case in which people used the correct signal structure S, but replaced the true distribution of Y by something else. A different belief formation rule is one that relies on the actual distributions and signal structure, but does not comply with Bayesian updating. A candidate is a belief that ignores the prior (or base rate) and overweights the likelihood. A recent strand of the literature systematically incorporates such deviations into belief formation in the form of diagnostic expectations (Bordalo et al., 217, 218). Since it is infeasible to identify and test every possible candidate rule, I proceed by ruling out categories of specifications based on the data. In an additional experiment, subjects faced various tasks that allow to distinguish between some of the main explanations. This evidence is reported in Appendix C.5. I make three observations. First of all, feature neglect empirically differs from likelihood-based explanations such as diagnostic expectations. People form diagnostic expectations if they overweight outcomes that become more likely in the light of new information (Bordalo et al., 218). However, in my data people typically overweight outcomes of X that are close to s, even if these outcomes have become less likely under s. For example, consider two independent, normally distributed variables X N (1, 1) and Y N (1, 1), and signal structure I = X + Y. Upon observing, e.g., s = 145, diagnostic expectations overweight small outcomes of X below 1. In the experiment, however, subjects overweight outcomes of X above 1, as if trying to explain the signal solely through X. Relatedly, empirical beliefs do not feature the kernel of truth property of diagnostic expectations, which implies that beliefs generally respond to news in a directionally correct, but excessive manner. In the experiment, subjects also respond to news that is fully uninformative about X. 22

24 Second, I find that noise neglect is not in line with belief formation using the correct information structure, but a modified prior about Y. Specifically, in several tasks, stated beliefs about X are not in line with any possible subjective belief distribution about Y on the union of the actual support of Y, the mean, median and mode of Y, and the number. This excludes any rule that replaces Y by a single value in its support, its mean, etc. as well as any rule that shrinks the variance of Y. Finally, I document the following patterns. If s is in the support of X, feature neglect corresponds to people overweighting the outcome(s) closest to s. If s is not in the support of X but sufficiently close, feature neglect corresponds to overweighting the outcomes in the support of X that are closest to s. If s is not in the support of X and sufficiently far from any value with positive likelihood, the share of feature neglect substantially decreases. 21 The main insight from this analysis is that, for the range of data and signal structures analyzed here, feature neglect is best characterized as a strong form of ignorance about the existence of Y. That means, people seem to apply a modified signal structure S i that is ignorant of Y. Of course, the above analysis abstracted from any changes in the relative frequency with which different updating rules were observed. Given the characterization of the form of feature neglect derived in this section, the study of the contextual drivers and underlying mechanisms for the observed updating rules is left to the following two sections. 4 The Role of Mental Representations: A Simple Framework 4.1 Motivation The sources of deviations from normatively optimal, Bayesian reasoning broadly fall into two categories. First, people may understand a problem correctly, but they do not engage in executing the necessary steps to arrive at a solution. This 21 These regularities should be interpreted with caution. First, the results pertain to the specific experimental design studied here, that is, algebraic signal structures in which X and Y are combined additively. In practice, information environments rarely have these features, let alone an explicit information structure. As such, the results above on how people deal with algebraically explicit information structure should not be overemphasized. 23

25 is typically associated with the presence of cognitive costs which lead people to avoid thinking effort (Caplin and Dean, 215; Fiske and Taylor, 213; Gabaix, 214; Shah and Oppenheimer, 28; Stanovich, 29). Second, people may not form a correct understanding of the situation to begin with, and so non-bayesian thinking is due to how they think about a problem. A recent strand of literature in economics examines the implications of misspecified models of reality (Bohren, 216; Bohren and D. Hauser, 217; Enke, 217; Enke and Zimmermann, 217; Esponda and Pouzo, 216; Eyster and Rabin, 214; Hanna et al., 214; Schwartzstein, 214). The latter work shares a common theme with the leading view in cognitive science, which emphasizes the role of internal representations for cognition, i.e., people s subjective model of the world (Clark, 213; Fodor and Pylyshyn, 1988; Newell and Simon, 1972). Representations determine how people mentally construe the world, e.g., which features a person attends to and processes. Importantly, they are the basis for subsequent computations within those representations that determine action and choice. In terms of the distinction made above, representations are tantamount to agents internal models that economics has recently started to incorporate, which precede the execution of any additional steps of reasoning that these models dictate. While knowledge about the structure and determinants of representations is limited, two central insights from cognitive science concern the role of initial representations, which are believed to form automatically. First, initial representations are characterized by (i) their automaticity, i.e. they emerge quickly and effortlessly, and are not subject to conscious control, (ii) they are geared to support a person in taking action rather than to truthfully mirror the external world, and are thus shaped by concrete task demands, and (iii) they tend to be simple, low-complexity models that economize on cognitive resources. Second, these representations are not easily overturned once they have formed. Instead, some form of cue is required to trigger a different representation. The concept of initial representations shares commonalities with the notion of an intuition-based System 1 that provides automatic, effortless responses to problems according to dual-process theories (Evans and Stanovich, 213; Kahneman, 23b). The representation-based account of cognition might provide a (partial) foundation for the structure of System 1 responses. Moreover, dual-process theories also feature the idea that overriding of the System 1 response by the deliberate, effortful System 2 does not occur automatically but relies on a triggering event, specifically, contextual cues (Kahneman, 23a; Stanovich and West, 28). 24

26 4.2 Conceptual Framework The following framework for belief updating takes guidance from the three main themes presented above: the distinction between representation and computation, the typical features of initial representations, and the need for cues to override initial representations. The purpose of this framework is (i) to provide a common structure for the body of evidence presented in this paper, and (ii) to set a foundation grounded in cognitive science for investigating further implications of this taxonomy. It is overtly informed and tailored to the environments studied in this paper and not intended as a general-purpose theory of belief updating. At the same time, the structure of the framework can be adapted to settings in other belief formation studies, as discussed below. An agent faces a belief updating problem given a signal. The basic framework can be summarized as follows: An initial default representation of the updating problem is formed automatically. It includes the set of states that the agent is incentivized to guess. In the presence of a cue, the agent switches to a complete representation. The agent picks a level of effort that determines whether he updates or sticks to the prior, based on trading off the expected reward against the cognitive cost. The agent states a belief b about the state of the world z Z. He maximizes expected utility max e [ E 1 ] 2 (b(e) z) 2 1 e (4) by choosing effort e. The stated belief has the following form: b }{{} Stated posterior = b o }{{} Prior +φ(e) [ m a FN }{{} Belief adjustment under feature neglect +(1 m) a B ] }{{} Bayesian belief adjustment (5) a denotes the optimal belief adjustment from the prior when incorporating the signal in Bayesian fashion (a B ) or under feature neglect (a FN, to be detailed below). Parameter m {, 1} takes value if the update is formed under a complete representation and is 1 under the default representation. The representation (or 25

27 mental model) m is fully determined by the presence of a cue, i.e., a non-empty set of available cues d. if d = m = 1 else Similarly, φ is an indicator function for whether the agent exerts sufficient effort C(a) to implement the update. Otherwise he ignores the signal and adheres to the prior. 1 if e C(a) φ = else In this formulation, the exogenous representation m pins down how a signal is incorporated, i.e., in Bayesian fashion or with feature neglect, and effort determines whether this update is formed or not. In the following[ I abbreviate] the expected loss function associated with a stated belief b as L(b) := E 2 1(b z) 1 2. The beliefs and effort levels associated with the agent s optimal strategy are characterized as follows. (6) (7) b + a B, C(a) if m = L(b + a B ) C(a) > L(b ) b, e = b + a FN, C(a) if m = 1 L(b + a FN ) C(a) > L(b ) b, else (8) Given a representation, the agent forms an update whenever the reduction in expected losses exceeds the cognitive costs associated with updating, and maintains the prior otherwise. I move on to characterize the elements of the framework with an eye toward the empirical analysis. The agent receives a signal generated by a known signal structure S : X Y S. 22 Z {X, Y, (X, Y)} is the vector of variables that features in the agent s loss function. For example, in treatment Narrow, Z = X, whereas Z = (X, Y) in Broad. The loss function is equivalent to a payoff rule, thus Z can be thought of as the vector of incentivized variables: given a stated belief, their realization changes the agent s payoff. The default representation. By default, the agent s problem representation only 22 Bold letters denote a variable s support. In the basic case, X, Y and S are real-valued, but the analysis is readily extended to multidimensional settings. 26

28 includes the dimensions Z that enter his loss function. If Z = X, as in Narrow, he ignores Y. 23 The findings about the nature of feature neglect indicate that different representations can be characterized by specific subjective signal structures. Specifically, under a complete representation, the subjectively perceived signal structure equals the true one, S i = S. Under the default representation, the agent forms a subjective signal structure that differs from the true one, S i = S D = S. In treatment Narrow with Z = X, S D : X S D. With real-valued X, Y and S, I have shown in Section 3.2 that S D is best characterized by S D = X. Belief adjustments under the default and complete representations. The representationinduced subjective signal structure pins down a belief adjustment a relative to the posterior. This adjustment is derived from the Bayesian posterior under the subjective signal structure, i.e., P(Z S i ) = P(S i=s X) P(X) P(S i =s). 24 Given the quadratic loss function, the agent states the expectation of this posterior belief, and the adjustment are characterized as follows. a B = E[Z S = s] b (9) a FN = E[Z S D = s] b (1) Cues. Cues trigger a reconsideration of the default representation and induce the agent to switch to the complete representation. Cues can take a variety of forms, such as (i) an explicit hint that directs attention to the neglected part of the problem, (ii) the action itself, which can serve as a reminder or pointer, e.g., when only X is incentivized, but the joint belief about X and Y is elicited, (iii) a self-generated hint due to a high ability for reflective thinking (Stanovich, 211) or explicit deliberation on the structure of the problem, and (iv) implausible or unlikely posterior beliefs formed under the default representation. The latter situation occurs, for example, if the subjective signal structure S D is incompatible with the observed signal value, P(S D = s) =. Section 3.2 already showed that this leads to a reduction of feature neglect More generally, the default representation might not include all dimensions of Z. For example, when Z is high-dimensional the default representation excludes the dimensions that affect the agent s payoff by a sufficiently small amount, L z j < l. This idea is similar to the sparse max of Gabaix (214). 24 Note that given a discrepancy between S and S D, it is possible that P(S D = s) =, i.e., the agent observes a signal realization with zero subjective likelihood and the above expression is not well-defined. This event can be a cue that triggers overriding of the default representation, see below. 25 The results further indicated a tendency to jump to nearby values of X whenever P(S D = s) =. Intuitively, subjects might allow for a small perceptual error ε such that P(S D = s + ε) > while maintaining their mental model. 27

29 Effort and cognitive cost. In this framework, conscious effort affects the implementation of an update, but not how representations emerge. Given a representation, an effort level is chosen so as to trade off the expected reward (or minimization of loss) against the cognitive cost. Regarding the former, note that subjects in Narrow will mistakenly predict a lower loss under the default representation than under the complete representation. This is because they take their subjective signal structure as the basis for prediction, and the default signal structure is generally more informative about X than the true one. The basic framework further assumes a model-independent cognitive cost of an adjustment, C(a). This is an implausible simplification: updating under a more complex subjective signal structure requires higher effort, something people are likely to take into account. A more realistic extension of the framework incorporates variable cognitive costs that are driven by the complexity of the updating problem. A reasonable proxy is the dimensionality of the domain of the subjective signal structure, e.g., C(a Si ) J {X,Y} S i J. 5 Mechanisms and Debiasing This section investigates the mechanisms underlying the observed updating patterns. The empirical analysis is guided by the framework developed in Section 4. An improved understanding of the cognitive processes associated with belief formation holds the promise to identify common primitives if they exist of the diverse collection of deviations from rationality documented by the heuristics and biases program (Fudenberg, 26). Moreover, it potentially helps to design policies that target specific mechanisms with what Handel and Schwartzstein (218) call mechanism policies. Appendix A provides a complete list of all experiments conducted for this study. 5.1 Incentives and Cues: Shifting Effort vs. Shifting Attention In this subsection, I test a key comparative static of the conceptual framework. The central intuition of the representation-based updating model is that representation and computation are distinct: while representations are to a large extent exogenous and depend on contextual factors and cues outside of the agent s control, computation or the execution of an updating strategy is primarily a result of conscious effort. Following this logic, the presence of feature neglect, which is presumably 28

30 driven by a flawed representation, will not respond to effort, but signal neglect will. Conversely, specific cues should have the ability to reduce feature neglect. Prediction 3. The frequency of feature neglect is reduced by cues, but not by higher effort induced by incentives. Signal neglect, by contrast, responds to incentives. The experiments presented here were conducted online using MTurk. They follow the same design and procedures that were introduced in Section 3.1. For all mechanism experiments the five tasks displayed in Table 4 were used. The results are summarized in Figure 5. For each treatment, I show the fraction of beliefs in line with each of the three updating rules as well the fraction of beliefs that do not accord to any of these updating rules, pooled across all five tasks. I compare the change in the prevalence of different updating rules relative to the baseline treatment. Table 4: Online tasks in mechanism treatments X Y I N (1, 1) N (, 1) X + Y N (1, 1) N (, 1) X + Y N (1, 1) N (, 4) X + Y U[75, 76,..., 125] U[ 25, 24,..., 25] X + Y X+Y U[75, 76,..., 125] U[9, 91,..., 11] 2 Notes: This table provides an overview of the five baseline belief tasks in the online mechanism experiments. Note that for all normally distributed variables, the support was discretized to integers, truncated at µ 5 and µ + 5 and then the distributions were scaled to have unit probability mass. In treatment High stakes, the stake size was increased five-fold relative to Baseline. Under higher incentives, I observe that the prevalence of Bayesian updating increases statistically significantly at the expense of information neglect. Strikingly, the share of feature neglect remains roughly constant. Both observations are in line with the conceptual framework. Moreover, subjects significantly increased effort as measured by response times, both overall and within each subgroup (all p <.1). This means, under high incentives, subjects tried harder, but that only affected non-updating, reducing the fraction of subjects that ignored the signal altogether. Higher effort did not make people less likely to commit feature neglect, however. This indicates that psychic 29

31 costs, cognitive miserliness, laziness or effort reduction are a potent explanation for signal neglect, but do not seem to be an important source of feature neglect. Compellingly, a tenfold increase of the stake size in the laboratory experiment lead to a similar pattern, see Appendix D.2. 6% Baseline High stakes Hint Deliberation time 4% Feature neglect Bayesian Information neglect Other 2% % 15% 16% 28% 41% 18% 4% 35% 43% 12% 23% 51% 14% 25% 13% 37% 25% Figure 5: Fraction of beliefs in line with feature neglect, Bayesian updating or information neglect, as well as the remaining share of beliefs, across four mechanisms treatment conditions. All stated beliefs, pooled across conditions. Error bars indicate standard errors of the proportion. Next, I examine the impact of the second type of intervention, contextual cues. The framework posits that due to the nature of the default representation, people fail to detect the need to account for Y to begin with, so that feature neglect is rooted in people s unawareness. To test this, treatment Hint is designed to shift people s mental model directly, by raising awareness about the necessity to account for Y. On every elicitation screen, subjects saw a statement that read Also think about the role of Y. The hint merely shifts attention to Y, but does not provide direct instructions on how to solve the updating problem. If subjects were aware of the role of Y in the updating problem from the beginning, this hint should not have an effect. It further does not change the cognitive costs associated with accounting for Y in processing the signal. Figure 5 documents a dramatic decrease in the fraction of feature neglect by 3

32 almost two thirds once the hint was added. At the same time, Bayesian updating rose to a share of roughly 5%. Without changing monetary incentives or cognitive costs, the hint had a substantial effect on the updating pattern, in line with the idea that subjects are in principle willing and able to update in Bayesian fashion, but simply fail to notice the need to account for Y to begin with. Again, this results supports the main intuition of the framework. Treatment Hint is the brute force method of broadening people s representation. Can subjects be cued to reconsider their default representation by themselves? Treatment Deliberation time is based on the supposition that while the evidence clearly indicated that higher stakes increased effort subjects exerted this additional effort at the wrong stage of the solution process. I suggest that a quick and automatic default representation came up which was taken as given, and higher stakes induced subjects to try harder at optimizing within this representation. Absent a cue, however, they failed to detect the need to challenge their solution strategy even under higher incentives. In treatment Deliberation time, subjects faced a 3-second waiting time on each elicitation screen, during which they could not enter a guess or submit the page. The input fields were activated after that time ran out. I predicted that this enforced waiting time nudged people to deliberate on their solution strategy, and potentially led them to recognize the need to account for Y. Figure 5 shows that this was clearly the case: The share of feature neglect in Deliberation time fell substantially, roughly by half as much as the effect size of a hint. Taking stock, I find that in line with the concept of simplified mental representations that imply unawareness about the need to account for Y, only interventions that directly altered this representation could reduce feature neglect. Cues made people switch to the correct mental frame and they predominantly arrived at Bayesian beliefs. Conversely, If people were paid more, they tried harder by increasing effort on the consciously executed steps of their updating strategy. However, this did not affect the prevalence of feature neglect as subjects operated within a flawed representation to begin with. By contrast, signal neglect did react to incentives and complied to the predictions of effort-related accounts based on cognitive costs. Result 3. The frequency of feature neglect was reduced by a hint and enforced deliberation, but not by higher stakes. Signal neglect decreased under higher stakes. 31

33 5.2 Responsiveness to the Information Structure as Opposed to Signal Realizations The conceptual framework harnessed the notion of default representations that originates in cognitive science. Default representations were characterized as being geared to taking action and thus the immediate task demands, and at the same time they obey the principle of parsimony and are typically not well attuned to the higher-level structure of the information environment. A direct implication is that the default representation tends to be insensitive to variations in context as long as the basic structure of the situation is maintained. Accordingly, the form of the default representation should be robust to changes in the data distribution and signal structure as long as the action and incentive structure remain the same. However, the default representation can be more harmful in some environments than others in that it implies to larger deviations from the Bayesian benchmark. The representation-based updating framework therefore predicts that the default representation is to some degree insensitive to the expected accuracy of its associated beliefs in different information environments. Does this imply that feature neglect does not respond to variation in the information environment? This is not the case because signal structures associated with greater distortions under feature neglect are also more likely to generate observed signals that are implausible or unlikely under the default signal structure. Such situations, as discussed in Section 4.2, can serve as cues that trigger a reconsideration of the default representation. This line of reasoning distinguishes between an ex-ante and an ex-post notion of responsiveness to an information structure. First, the propensity to commit feature neglect does not per se respond to its anticipated accuracy in an information environment. That means, for a given signal value, the share of feature neglect should not differ across slight variation in the information environment. Second, we still expect to see less feature neglect on average in because people will receive more implausible signal in information environments where signal neglect is implausible, which trigger overriding of the default representation. I tested these predictions in additional experiments in which information structure was varied in systematic ways. Across tasks, the signal structure was held fixed. However, the amount of variance (Study 1) and bias (Study 2) introduced into the signal structure by Y varied across tasks. Intuitively, as the signal-to-noise ratio σx 2 σy 2 increases, a neglect of Y induces fewer distortions. Similarly, as the directional 32

34 bias E[S] E[X] increases due to a shift in the mean of Y, neglecting Y leads to increasingly distorted beliefs about X. Prediction 4. The frequency of feature neglect decreases with higher noise and directional bias introduced in the signal structure by Y. There is no effect, however, conditional on observing identical signal realizations. Note that this prediction of the conceptual framework differs substantively from models in which attention is chosen rationally based on expectations about the costliness of specific simplifications (Caplin and Dean, 215; Gabaix, 214) Study 1: Treatment Signal-to-Noise Ratio Treatment Signal-to-noise ratio varied the ratio between the variance of X, σx 2, and the variance of Y, σy 2. I ran an additional online experiment with seven tasks as shown in Table 5. A sample of N = 29 participated in this experiment, where again task order was randomized and one task was randomly incentivized with a maximum prize of 3 dollars. Table 5: Online tasks: Experiment on signal-to-noise ratio X Y S λ = σ2 X σx 2 +σ2 Y N (1, 25) N (, 16) X + Y.15 N (1, 1) N (, 16) X + Y.59 N (1, 25) N (, 1) X + Y.25 N (1, 1) N (, 1) X + Y.5 N (1, 1) N (, 25) X + Y.75 N (1, 16) N (, 1) X + Y.941 N (1, 16) N (, 25) X + Y.985 Notes: This table provides an overview of the five tasks in the online experiment on the effect of the signal-to-noise ratio. Note that for all normally distributed variables, the support was discretized to integers, truncated at µ 5 and µ + 5 and then the distributions were scaled such that the they have unit probability mass. m 1 s 1 Figure 6 documents the results by plotting estimated kernel densities of ˆλ = by task. In line with the previous results, there are three empirical modes in each task, corresponding to Bayesian updating, feature neglect and signal neglect. 33

35 Note that the value of λ in line with Bayesian beliefs changes across tasks, as indicated by the dashed diagonal line. To support the visual analysis, I perform non-parametric test on the distributions of ˆλ. First, the summed share of beliefs in line with either one of the three updating modes (defined as being within [λ.5, λ +.5]) does not significantly differ across tasks (p >.1 for all pairwise comparisons in χ 2 tests). Second, for each task with λ >.75, the share of beliefs in line with feature neglect (again, defined as being within [.95, 1.5]), is significantly higher than in all tasks with λ <.75 (all p <.5, pairwise χ 2 tests). Third, for each task with λ <.25, the share of beliefs in line with signal neglect (defined as being within [.5,.5]), is significantly higher than in all tasks with λ >.25 (all p <.1, pairwise χ 2 tests). This means that, in line with the prediction, the share of feature neglect increases with increasing signal-to-noise ratio λ. Note, however, that as the signal-to-noise ration decreases, subjects are also more likely to observe signal realizations further away from 1, which is the mean of the normally distributed X. This means the subjective likelihood of a signal P(S i = s) decreases. In additional analyses reported in Appendix D.1, I show that while feature neglect increases in λ, this effects becomes insignificant controlling for the subjective likelihood of observed signal values under feature neglect. That means, in two tasks with identically distributed X and an identical observed information value s, there is no statistically significant difference in the propensity to commit feature neglect Study 2: Treatment Directional Bias The second treatment, Directional Bias, examined the effect of directional bias in information while fixing the signal-to-noise ratio. Under feature neglect, beliefs are more biased on average the larger the deviation between the mean signal value and the mean of X. The five task configurations of this online experiment are displayed in Table 6. Note that λ = σ2 X σ 2 X +σ2 Y is identical across tasks, setting this experiment apart from treatment Signal-to-noise ratio. However, the expected value of the information, µ S, varies. To optimally learn from S, subjects need to account for the fact that observed values of S are on average higher or lower than the corresponding draws of X if µ X = µ S. Feature neglect as conceptualized in the framework is blind to this change in the information environment 34

36 λ^ λ Figure 6: Kernel density estimates for seven different tasks (see Table 5) in an online experiment testing the effect of the signal-to-noise ratio on the prevalence of different updating modes. The horizontal axis indicates λ = σ2 X σ 2 X +σ2 Y of the task. The vertical axis shows the empirical equivalent derived from subjects guesses as ˆλ = m 1 s 1. Note that ˆλ = indicates signal neglect, ˆλ = 1 indicates feature neglect, and the dashed line indicates the ˆλ that corresponds to Bayesian updating. Based on N=27 in each task. Epanechnikov kernel with bandwidth.7. Raw beliefs for each task are plotted in Figure 7. The figure indicates that subjects were less likely to commit feature neglect as the directional bias of the signal increased, i.e., the greater the distance of µ Y from. This observation is supported by non-parametric tests. The share of beliefs in line with feature neglect significantly decreased as µ S moved away from Notably, 26 That is, the share of feature neglect decreased in both directions away from 1 for adjacent tasks, e.g., both for µ S = 1 vs. µ S = 75 and µ S = 75 vs. µ S = 5. Feature neglect can be defined in different ways. I either define it as the any guess falling within a margin of 5 units around the hypothetical feature neglect guess, or based on d rel d FN falling within.5 to either side of FN = d FN +d B +d SN, where d is the distance of a stated belief m to the respective benchmark belief for each of three updating modes. That means, e.g., d B = m mb is the distance to the Bayesian belief. Hence, d rel FN is the distance of a belief to a hypothetical belief under feature neglect, relative to the summed distances of the elicited belief to all three updating modes. p <.5 in all pairwise χ 2 tests. 35

37 Table 6: Online tasks: Experiment on directional bias in information X Y I µ I N (1, 1) N (, 1) X + Y 1 N (1, 1) N ( 25, 1) X + Y 75 N (1, 1) N ( 5, 1) X + Y 5 N (1, 1) N (25, 1) X + Y 125 N (1, 1) N (5, 1) X + Y 15 Notes: This table provides an overview of the five tasks in the online experiment on the effect of the directional bias in signals. Note that for all normally distributed variables, the support was discretized to integers, truncated at µ 5 and µ + 5 and then the distributions were scaled such that the they have unit probability mass. 15 X = N(1,1) Y = N(,1) S = X+Y X = N(1,1) Y = N( 25,1) S = X+Y X = N(1,1) Y = N(25,1) S = X+Y Guess about X 15 X = N(1,1) Y = N( 5,1) S = X+Y X = N(1,1) Y = N(5,1) S = X+Y Signal Bayesian posterior Noise neglect Signal neglect Figure 7: Beliefs in baseline tasks of online experiments. N=112 in each task. Each dot corresponds to one stated belief. The three red line indicate the Bayesian benchmark, noise neglect, and information neglect. I found that this decrease went hand in hand with an increase in Bayesian beliefs, 36

38 rather than signal neglect. 27 Another way to illustrate this result is to characterize an observed belief by how relatively close it is to each of the three benchmarks. I do this by computing the distance to each of the three benchmarks, then add up these distances, and calculate a measure of relative proximity for each updating rule as the fraction of the distance to that rule relative to the sum of distances. Specifically, I obtain three measures of relative distance for each elicited belief, all of which lie between, meaning the belief corresponds exactly to the posterior implied by that updating rule, and 1. These three measures sum up to 1. A ternary plot of beliefs characterized by these three distances is shown in Figure 8. In each triangle, the distance from the posterior under feature neglect corresponds to the vertical distance from the bottom. That means, a belief close to the horizontal axis is in line with feature neglect, while are belief further away from it indicates a larger distance from noise neglect. Similar to the case of treatment Signal-to-noise ratio, however, as directional bias in a signal structure increases, subjects are more likely to observe signal values with low subjective likelihood under feature neglect. This means the plausibility of observed signals decreases under the subjective signal structure associated with the default representation, and hence a signal is more likely to work as a cue. In the regression analyses reported in Appendix D.1, I again find that the decrease of feature neglect with increasing directional bias vanishes after controlling for the subjective likelihood (assuming feature neglect) of observed signal values. Result 4. The overall share of feature neglect decreases as noise or directional bias in a signal structure increase, but this effects is purely driven by the associated difference in observed signal realizations. Conditional on seeing the same signal, a different information structure has no effect on the propensity to commit feature neglect. 5.3 Are Subjects Aware Of Feature Neglect? An important characteristic of representations is the lack of awareness about the features of the environment that are not represented. Because the representation structures perception by selecting the elements that receive attention, all other, unat- 27 The share of Bayesian beliefs as defined above significantly increases with the distance of µ S from 1, p <.1 in all pairwise χ 2 tests. 37

39 Y = N( 5,1) Y = N( 25,1) Y = N(,1) Distance to Bayesian Distance to noise neglect Distance to info neglect Distance to Bayesian Distance to noise neglect Distance to info neglect Distance to Bayesian Distance to noise neglect Distance to info neglect 2 1 # of obs. 6 Y = N(25,1) Y = N(5,1) Distance to Bayesian 2 8 Distance to noise neglect Distance to Bayesian 2 8 Distance to noise neglect Distance to info neglect Distance to info neglect 1 Figure 8: Beliefs in online experiments on the effect of directional bias in information. N=112 in each task. Each point corresponds to one stated belief. Red areas in the heatmap indicate regions with more stated beliefs. The displayed data is computed based on three measures of relative distance for each belief that sum to one, see description in main text. tended elements are not processed to begin with and thus do not form part of the internal model of the external world. To test this intuition, I examine people s awareness about feature neglect directly by measuring confidence in their stated beliefs. If subjects had some form of awareness about the need to account for Y even under feature neglect, they should be less confident in the resulting belief. If instead this neglect is driven by representation and occurs outside of awareness, it is conceivable that subjects deliberately and purposefully execute the subsequent computations, still leading to high confidence in the result. Prediction 5. Subjects are not aware about feature neglect and therefore fully confident in their stated beliefs. Design. Two additional experimental variations in the laboratory directly ex- 38

40 amine subjects metacognition of inference, i.e., what they think about their own solution strategy in the belief updating tasks. In stage Confidence following the baseline belief tasks, subjects indicate their willingness-to-accept for each previously stated belief. To this end they again see each individual updating task in combination with their own stated belief. They then indicate whether they prefer to be paid out for their belief based on the scoring rule or receive a fixed monetary amount. Subjects make this binary decision for different fixed amounts ranging from euros to 6 euros, presented in a multiple-price list format. In case this task would be chosen for payoff, their decision in one of the rows of the list would be implemented. Note that the Confidence tasks (i) had no time limit such that subjects could freely rethink their stated belief, and (ii) the subjective valuation in each task provides a measure of confidence in the belief itself, beyond the variance implied by the stated belief distribution. In stage Switch-role at the end of the laboratory baseline experiment, each subject played two bonus rounds in the opposite condition. Are participants in Broad, who have previously formed Bayesian beliefs in these tasks, able to transfer their successful solution strategy to an updating problem with narrow incentives that is otherwise identical? This requires a metacognitive understanding of one s own previously implemented solution. Results. Columns 1 to 4 of Table 7 present results from the Confidence tasks using regressions in which the dependent variable is the subjective valuation of a stated believe, i.e., the minimal certain amount preferred over a having the stated belief paid out. A higher value corresponds to more confidence. Strikingly, more inattentive beliefs are not significantly associated with lower reservation prices. Even after reconsidering the updating problem and their own belief, subjects fail to recognize the necessity to account for Y and are equally confident in their own guess. Reassuringly, the variance of the indicated belief distribution negatively affects confidence. Restricting the sample to beliefs stated in Narrow, there is again no relationship between the valuation of a stated belief and implied inattention. These results suggest that inattentive inference relies on processes outside of a person s awareness. In columns 5 to 7 of Table 7 I analyze scores of inattention to noise on the pooled sample of beliefs from the baseline and Switch-role tasks. I find that (i) unsurprisingly, Narrow subjects almost immediately improve when facing the broad setup, and display a similar level of inattention as Broad subjects in the baseline 39

41 (p >.7, see footer of Table 7), (ii) Broad subjects do transfer their experience in forming Bayesian beliefs, as indicated by a significant improvement relative to Narrow subjects in baseline (p <.5), (iii) this transfer, however, is far from perfect and a significant treatment effect between Narrow and Broad persists in the Switch-role tasks, albeit now with the reverse sign. In fact, mean inattention in Broad is.59 in Switch-role, compared to baseline means of.11 in Broad and.73 in Narrow. Put differently, the improvement in Broad is marginal and subjects effectively commit inattentive inference to a roughly similar extent as if they had not had the baseline experience. In sum, the combined evidence clearly suggests that the psychological mechanism responsible for inattentive inference operates outside of people s awareness. Selective processing of the features of an information environment shaped by the structure of the prediction incentives leads to a narrow representation of the problem. The subsequent cognitive computations are executed deliberately and confidently but rely on a flawed mental model of the environment. 4

42 Table 7: Mechanisms underlying inattentive updating: Awareness about the problem structure Dependent variable: Confidence: Valuation for stated belief Inattention θ Condition: Narrow and Broad Narrow Narrow and Broad Tasks: Baseline and robustness Baseline and switch-role (1) (2) (3) (4) (5) (6) (7) if Broad, 1 if Narrow ***.616***.616*** (.316) (.317) (.3) (.47) (.47) (.46) Inattention θ (.59) (.58) (.512) (.436) Treatment dummy * Inattention θ (.62) (.619) (.612) Variance of belief distribution -.*** -.* -. (.) (.) (.2) Willingness to take risks.555***.645*** (.134) (.175) if main task, 1 if reverse task.484***.442***.443*** (.52) (.56) (.56) (1 if Narrow) * (1 if switch-role task) -1.45*** -1.43*** -1.45*** (.69) (.69) (.69) Constant 4.55*** 4.555*** 3.694*** 3.11***.11***.126*** -.7 (.18) (.181) (.637) (.613) (.24) (.31) (.54) Task fixed effects Yes Yes Yes Yes Additional controls Yes Yes Yes i) Mean inattention Broad, baseline ii) Mean inattention Narrow, switch-role i) vs. ii): F 1, iii) Mean inattention Narrow, baseline iv) Mean inattention Broad, switch-role iii) vs. iv): F 1, ** 6.99*** 6.71** R # Observations Notes: OLS regressions. Inattention to noise is calculated as θ = H B H B +H N, where H B and H N denote the Hellinger distance of the subjective distribution to the Bayesian posterior and the inattentive posterior distribution, respectively. Robust standard errors clustered at participant level in parentheses. The switch-role task is the final, additional belief task in which we switched experimental conditions, i.e., subjects in Broad had to guess only X and subjects in Narrow guess both X and Y. The additional controls include gender, age, income and task-fixed effect. Group means of inattention and tests of the differences in group means are reported in the footer. Robust standard errors clustered at participant level in parentheses. *p <.1, **p <.5, ***p <.1. Result 5. Feature neglect is associated with confidence ratings in beliefs that are similar to those under Bayesian updating, suggesting that subjects are unaware about the neglect. 41

43 5.4 Limits to Learning and the Persistence of Biases Prima facie, it is puzzling that the reported updating errors arise despite the substantial costs associated with them. Being permanently exposed to compound information, why do we not learn to adequately incorporate them into our beliefs? To date, there is no consensus view on the sources of persistence of heuristics and biases (e.g., Gigerenzer, 1991; Stanovich and West, 2). There limits of learning from feedback are apparent, but their reasons are not well understood. The conceptual framework and previous results suggest a channel for limits of learning based on awareness: if people learn from surprising feedback by reflecting on their own solution process, they will first and perhaps exclusively address those elements that are available to introspection. A central feature of the representation-based account of updating is that the processes responsible for the emergence of representations are implicit and inaccessible to the conscious mind. They are unavailable to introspection and recall. This is why they cannot be actively targeted by the learner. Computational components of a solution process, however, are typically accompanied by some metacognitive experience (Ackerman and Thompson, 217) and come to mind given surprising feedback. These computations on the representation are more likely to be targeted in the learning process. But they are not the source of the error in the case of inattentive inference. Under this hypothesis, errors due to unconscious processes can survive even in the presence of feedback. 28 A comparative static emerging from this line of reasoning is that the propensity to learn from feedback is shaped by the presence of conscious computational steps in the solution process. The more conscious computations are involved in forming a response under feature neglect, the less likely it will be that surprising feedback induces a person to challenge his internal representation, holding fixed the complexity of the problem. I designed additional treatments to directly test this prediction. Prediction 6. The extent of learning about feature neglect from surprising feedback is limited, because subjects tend to reflect on the conscious computations of their solution process, but fail to challenger their representations. 28 By feedback I mean all types of feedback that are unspecific to the specific neglect committed. An explicit hint to Y allows to exert executive attention and directly influence the percept. 42

44 Design. Treatment Feedback is akin to Narrow, but further provides the most natural unspecific type of feedback in this setting. In each of the five baseline tasks, after guessing X, subjects see the true value of X. Under fully inattentive inference, the true value can subjectively be a zero probability event. Still, I expect limited improvement across tasks, because subjects may not get to the point of reflecting on their neglect of Y. Next, to create exogenous variation in the extent of consciously accessible reasoning, condition Computation with Feedback inserts a simple algebraic computation to the signal structure, which is identical to that in Computation. 29 Recall that these computations are extremely simple, e.g., Notably, standard accounts of dual processing consider these simple algebraic problems as recruiting Type 1 reasoning, because the answer suggests itself without intention (Evans and Stanovich, 213; Thompson, 213). A key distinction to the nature of the processes driving inattentive inference is that it creates a metacognitive experience, that is we are aware of somehow having produced a result. The results in Computation (Section 2.6) showed that the computation is inconsequential for the guesses about X that subjects actually submit. Presented with surprising feedback about the actually drawn number, however, subjects in Computation with Feedback might recall the conscious part of their inference strategy, i.e., undoing the calculation. The computation provides an obvious albeit unlikely source of error. I hypothesize that this significantly reduces learning relative to the Feedback baseline. Note that reduced learning when computations are added can also be the result of increased complexity. Adding more sources of error can reduce the likelihood to question each individual one of them in the learning process. I therefore design a third treatment, Computational Feedback, in which (i) subjects receive the same feedback as before, (ii) the inference problem including the signal structure is identical to the no-computations case in Feedback, and (iii) the setting features identical computational complexity to Computation with Feedback. Now, if reduced learning in Computation with Feedback is solely the result of increased complexity, one should expect the same in Computational Feedback. If, however, learning is only compromised by computations performed when doing inference, we would not expect to see reduced learning here, since the inference problem is identical to Feedback. I predict the latter. 29 That is, Computation with Feedback is identical to condition Computation, except for the feedback; and it is identical to condition Feedback, except for the simple computation that needs to be undone. 43

45 In Computational Feedback, subjects receive a signal on X and Y without additional computations, i.e., the mean or sum as before. This time, however, these same computations are added at the feedback stage. That means, instead of observing the true value of X, subjects see a different value on which they first need to perform the computations to arrive at the true value of X. Here, the computations are clearly executed after stating a guess about X, i.e., after inference. Seeing a surprising true value of X, now, subjects presumably recall that they performed the calculations when provided with the feedback, and that they earlier on indicated a guess, which itself was independent of these computations. That is, the computations are not directly associated with the inference process. Since the algebraic calculations are extremely simple, I expect subjects are instead somewhat more likely to reflect on the inference stage. Results. In the first round before receiving feedback for the first time, inattention is expectedly indistinguishable across the feedback treatments (see Appendix Figure A12). 3 I now analyze inattention scores of beliefs stated after feedback has been received in preceding rounds. In what follows, I restrict my attention to the fifth and last round, since learning effects should be highest after several rounds of feedback. 31 For ease of exposition, Figure 9 depicts mean inattention by treatment condition. All statistical analyses, however, are based on empirical distributions of inattention. 32 Inattention in the three relevant no-feedback conditions is displayed above the dashed horizontal line for comparison. I document three key findings. First, the provision of feedback alone (condition Feedback) significantly decreases inattention relative to Narrow (p <.1, M-W U test). At the same time, learning is far from perfect, as indicated by a remaining treatment effect between Feedback and the Broad benchmark without feedback (p <.1). That means, feedback about the actually drawn X generates substantial improvements, but it is no guarantee that subjects figure out their neglect of Y. Second, I find clear evidence for the hypothesis that additional computations in the inference process reduce learning. Inattention in Feedback and Computation with Feedback significantly differ (p =.8). This effect presumably operates by diminishing subjects propensity to realize that there are parts of the problem that they 3 This again shows that the computation added to the signal does not influence stated beliefs. 31 The following findings persist if beliefs from rounds two to five are pooled together. See further results in Appendix E. 32 See Appendix E for distribution plots. 44

46 Narrow Broad Computation Feedback Computation with Feedback Computational Feedback Inattention (mean ± s.e.m.) Figure 9: Treatment means of inattention to Y. The three treatments above the dashed line show conditions without feedback for reference. Feedback is about the truly drawn value of X. Displayed are implied inattention scores in the final baseline round, after having received feedback in the four preceding rounds. Sample sizes are N = 72 in both Narrow and Broad, N = 48 in Feedback, and N = 24 in each of Computation, Computation with Feedback and Computational Feedback. have not attended to. Notably, the addition of simple algebraic computation virtually eliminated learning. Inattention in Computation with Feedback is not significantly lower than in Computation (p =.21). Third, the documented reduction in learning is not driven by an increase in complexity. In fact, inattention in Computational Feedback is indistinguishable from Feedback (p =.48), but significantly differs from Computation with Feedback (p =.5) in the predicted direction. Result 6. Learning from feedback is targeted to the consciously executed computations in a solution process. This reduces overcoming feature neglect as soon as a problem requires computations. 45

47 Taking stock, the data clearly suggest that subjects learn to account for Y only if they cannot attribute prediction mistakes to any steps in the reasoning process that are available to introspection. In an attempt to do gather direct evidence of this hypothesis, the design includes an additional choice in all feedback treatments. On the feedback screen that informs about the actual draw, subjects could choose to be reminded of up to exactly one aspect of the preceding belief task: the distribution of X, the distribution of Y, or the signal structure. Revealing such details can help figure out the source of an erroneous guess and make better subsequent guesses. In the first round, i.e., upon receiving feedback for the first time, subjects are indeed more likely to reveal the distribution of Y in Feedback than in Computation with Feedback (p =.44). This effect, however, is not robust and loses significance when pooling all rounds. Procedural details and further results are relegated to Appendix E Conclusion This paper reports evidence from more than twenty experimental treatments, showing that people neglect alternative causes when updating from information structures with multiple causes. It concludes that this neglect is most likely rooted in the systematic structure of people s automatic, internal representations of an updating problem. 33 Finally, in practice available signals are usually noisy or imprecise. The possibility that observed feedback is not exactly right might provide another obvious way for subjects resolve the conflict between their subjective belief and feedback received, again reducing learning. In condition Imperfect Feedback, subjects are given feedback about the true X that is correct only with 8% probability, but would see a value of X which is not the true one with 2% probability. Again, learning is reduced in a similar way as by adding the computation, see Appendix E.4. 46

48 References Ackerman, Rakefet and Valerie Thompson (217): Meta-reasoning: Shedding meta-cognitive light on reasoning research. In. International Handbook of Thinking & Reasoning. Psychology Press (cit. on p. 42). Bandyopadhyay, Prasanta, Gordon Brittan, and Mark Taper (216): Belief, Evidence, and Uncertainty: Problems of Epistemic Inference. Springer International (cit. on p. 13). Barron, Kai, Steffen Huck, and Philippe Jehiel (218): Everyday econometricians: Selection neglect and overoptimism when learning from others (cit. on p. 3). Bertrand, Marianne and Sendhil Mullainathan (21): Are CEOs rewarded for luck? The ones without principals are. The Quarterly Journal of Economics, 116 (3), (cit. on p. 1). Bock, Olaf, Ingmar Baetge, and Andreas Nicklisch (214): hroot: Hamburg registration and organization online tool. European Economic Review, 71, (cit. on p. 1). Bohren, Aislinn (216): Informational herding with model misspecification. Journal of Economic Theory, 163, (cit. on pp. 3, 24). Bohren, Aislinn and Daniel Hauser (217): Social Learning with Model Misspecification: A Framework and a Robustness Result (cit. on pp. 3, 24). Bordalo, Pedro, Nicola Gennaioli, Rafael LaPorta, and Andrei Shleifer (217): Diagnostic expectations and stock returns. Tech. rep. National Bureau of Economic Research (cit. on p. 22). Bordalo, Pedro, Nicola Gennaioli, and Andrei Shleifer (218): Diagnostic expectations and credit cycles. The Journal of Finance, 73 (1), (cit. on pp. 22, 15). Cacioppo, John T, Richard E Petty, Jeffrey A Feinstein, and W Blair G Jarvis (1996): Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological bulletin, 119 (2), 197 (cit. on p. 6). Caplin, Andrew and Mark Dean (215): Revealed preference, rational inattention, and costly information acquisition. The American Economic Review, 15 (7), (cit. on pp. 4, 24, 33). Caplin, Andrew, Mark Dean, and Daniel Martin (211): Search and satisficing. The American Economic Review, 11 (7), (cit. on p. 8). Charness, Gary and David I Levine (27): Intention and stochastic outcomes: An experimental study. The Economic Journal, 117 (522), (cit. on p. 1). Chen, Daniel L., Martin Schonger, and Chris Wickens (216): otree An opensource platform for laboratory, online, and field experiments. Journal of Behavioral and Experimental Finance, 9, (cit. on pp. 1, 19). Clark, Andy (213): Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and brain sciences, 36 (3), (cit. on pp. 3, 24). 47

49 Dean, Mark and Nathaniel Neligh (217): Experimental tests of rational inattention. Tech. rep. Columbia University (cit. on p. 8). Enke, Benjamin (217): What you see is all there is. Mimeo (cit. on pp. 3, 8, 24). Enke, Benjamin and Florian Zimmermann (217): Correlation Neglect in Belief Formation. The Review of Economic Studies (cit. on pp. 3, 8, 24). Esponda, Ignacio and Demian Pouzo (216): Berk Nash equilibrium: A framework for modeling agents with misspecified models. Econometrica, 84 (3), (cit. on pp. 3, 24). Evans, Jonathan and Keith E. Stanovich (213): Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8 (3), (cit. on pp. 24, 43). Eyster, Erik and Matthew Rabin (214): Extensive imitation is irrational and harmful. The Quarterly Journal of Economics, 129 (4), (cit. on pp. 3, 24). Falk, Armin, Anke Becker, Thomas J. Dohmen, David Huffman, and Uwe Sunde (216): The preference survey module: A validated instrument for measuring risk, time, and social preferences. Mimeo (cit. on p. 1). Fiske, Susan T and Shelley E Taylor (213): Social cognition: From brains to culture. Sage (cit. on p. 24). Fodor, Jerry A and Zenon W Pylyshyn (1988): Connectionism and cognitive architecture: A critical analysis. Cognition, 28 (1-2), 3 71 (cit. on pp. 3, 24). Fudenberg, Drew (26): Advancing beyond advances in behavioral economics. Journal of Economic Literature, 44 (3), (cit. on pp. 3, 28). Gabaix, Xavier (214): A sparsity-based model of bounded rationality. The Quarterly Journal of Economics, 129 (4), (cit. on pp. 4, 24, 27, 33). Gigerenzer, Gerd (1991): How to make cognitive illusions disappear: Beyond heuristics and biases. European Review of Social Psychology, 2 (1), (cit. on p. 42). Gurdal, Mehmet Y, Joshua B Miller, and Aldo Rustichini (213): Why blame? Journal of Political Economy, 121 (6), (cit. on p. 1). Handel, Benjamin and Joshua Schwartzstein (218): Frictions or Mental Gaps: What s Behind the Information We (Don t) Use and When Do We Care? Journal of Economic Perspectives, 32 (1), (cit. on pp. 3, 28). Hanna, Rema, Sendhil Mullainathan, and Joshua Schwartzstein (214): Learning through noticing: Theory and evidence from a field experiment. The Quarterly Journal of Economics, 129 (3), (cit. on pp. 3, 24). Hauser, David J. and Norbert Schwarz (216): Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48 (1), 4 47 (cit. on p. 18). Hellinger, Ernst (199): Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. Journal für die reine und angewandte Mathematik, 136, (cit. on p. 13). Hossain, Tanjim and Ryo Okui (213): The binarized scoring rule. The Review of Economic Studies, 8 (3), (cit. on p. 7). 48

50 Kahneman, Daniel (23a): A perspective on judgment and choice: mapping bounded rationality. American psychologist, 58 (9), 697 (cit. on p. 24). Kahneman, Daniel (23b): Maps of bounded rationality: Psychology for behavioral economics. The American Economic Review, 93 (5), (cit. on p. 24). Khaw, Mel Win, Luminita Stevens, and Michael Woodford (217): Discrete adjustment to a changing environment: Experimental evidence. Journal of Monetary Economics, 91, (cit. on p. 8). Newell, Allen and Herbert A. Simon (1972): Human Problem Solving. Prentice-Hall (cit. on pp. 3, 24). Paulus, Christoph (29): Der Saarbrücker Persönlichkeitsfragebogen SPF (IRI) zur Messung von Empathie: Psychometrische Evaluation der deutschen Version des Interpersonal Reactivity Index (cit. on p. 1). Rammstedt, Beatrice and Oliver P. John (25): Kurzversion des big five inventory (BFI-K). Diagnostica, 51 (4), (cit. on p. 1). Raven, Jean et al. (23): Raven progressive matrices. In Handbook of nonverbal assessment. Springer, (cit. on p. 6). Schwartzstein, Joshua (214): Selective attention and learning. Journal of the European Economic Association, 12 (6), (cit. on pp. 3, 24). Shah, Anuj K and Daniel M Oppenheimer (28): Heuristics made easy: an effortreduction framework. Psychological bulletin, 134 (2), 27 (cit. on p. 24). Stanovich, Keith E. (29): What intelligence tests miss: The psychology of rational thought. Yale University Press (cit. on pp. 5, 24). Stanovich, Keith E. (211): Rationality and the reflective mind. Oxford University Press (cit. on p. 27). Stanovich, Keith E. and Richard F. West (1997): Reasoning independently of prior belief and individual differences in actively open-minded thinking. Journal of Educational Psychology, 89 (2), 342 (cit. on p. 5). Stanovich, Keith E. and Richard F. West (2): Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23 (5), (cit. on p. 42). Stanovich, Keith E. and Richard F. West (28): On the relative independence of thinking biases and cognitive ability. Journal of personality and social psychology, 94 (4), 672 (cit. on pp. 5, 24). Thompson, Valerie (213): Why it matters: The implications of autonomous processes for dual process theories Commentary on Evans & Stanovich (213). Perspectives on Psychological Science, 8 (3), (cit. on pp. 43, 9). Wolfers, Justin et al. (22): Are voters rational?: Evidence from gubernatorial elections. Graduate School of Business, Stanford University (cit. on p. 1). 49

51 Appendix 1

52 A Overview of Treatments Table A1: Overview of laboratory treatments Condition Description Covered in Baseline experiment: Narrow and Broad Baseline Tasks Robustness Tasks Bonus Task Confidence Switch-role Computation Simplification (Elements of baseline experiment in respective order below) 5 updating tasks in random order. X and Y follow independent discrete uniform distributions with outcome spaces smaller than 1. The information is the mean or the sum of the draws. 5 updating tasks in random order. Data are correlated, drawn from a larger sample space, discretely normally distributed, or the information is outside of the range of X. 1 surprise task with similar configuration to baseline. Within each condition, subjects are re-randomized and face either the same expected incentive size as before, or tenfold incentives. For each baseline and robustness problem, subjects indicate their valuation for their stated belief using a multiple-price list method. 2 tasks with similar configuration as baseline, but subjects face incentives from opposite treatment condition. That means Narrow is paid for X and Y, while Broad paid for X only. Identical to Narrow baseline, except that a simple, task-varying algebraic calculation is added to the information structure (e.g., the mean +2 3 ). Identical to Narrow baseline, but deciphering stage and all time limits removed. Appendix B Appendix C.1 Main text Appendix D.4 Appendix D.5 Appendix C.2 Appendix C.4 2

53 Condition Description Covered in Narrow with joint elicitation Broad with sequential elicitation Hint Feedback Identical to Narrow baseline, but subjects indicate the joint distribution of X and Y (while only X is paid for). Identical to Broad baseline, but subjects indicate the marginal distributions of X and Y in sequential order, such that the first screen is identical to Narrow baseline. Identical to Narrow baseline, but subjects receive a reminder on the elicitation screen, stating Also think about the role of Y. Identical to Narrow baseline, but subjects observe the actual draw of X after stating their guess. Appendix C.3 Appendix C.3 Appendix D.3 Appendix E.1 Computation with Identical to Computation, but subjects observe the Appendix feedback actual draw of X after stating their guess. E.2 Computational feedback Imperfect feedback Identical to Computation with feedback, except that the computation is added to the feedback instead of the information. Identical to Feedback, but subjects receive the true draw as feedback only with 8% probability, while seeing another value with 2% probability. Appendix E.3 Appendix E.4 3

54 Table A2: Overview of online treatments Condition Description Covered in Baseline experiment (Narrow only) Baseline for mechanisms experiments 5 updating tasks in random order (Table 3). X and Y follow independent distributions. Subjects only state a mean posterior belief about X. No deciphering stage. 5 updating tasks in random order (Table 4) followed by one confidence tasks in which subjects indicate their WTA for a stated belief. Otherwise identical to online baseline experiment. Main text Main text High Stakes As mechanism baseline with fivefold incentives. Main text Hint As mechanism baseline, but additional hint: Also think about the role of Y. Main text Hint and High Stakes Combination of treatments Hint and High Stakes Appendix Deliberation time As mechanism baseline, 3-second waiting time enforced before input forms activate and page can be submitted. Main text Deliberation and Combination of treatments Deliberation Time and Appendix High Stakes High Stakes Form of Noise Neglect 1 updating tasks in random order (Table A5). Identical to baseline online experiment but different information structures to analyze different candidates for the belief formation rule under noise neglect. Main text and Appendix C.5 Signal-to-Noise Ra- 7 updating tasks in random order. Identical to Main text tio baseline online experiment but different information structures (Table 5). The signal-to-noise ratio is varied between tasks. Directional Bias 5 updating tasks in random order. Identical to base- Main text line online experiment but different information structures (Table 6). All elements of the information structure are kept fixed across tasks except the mean of Y. 4

55 B Baseline Experiments B.1 Procedure of Updating Tasks: Laboratory Experiment Learn joint prior distribution of random numbers X and Y. Learn signal structure: sum or average, depending on task. Receive encrypted signal, i.e. a sequence of letters. Decipher signal using algorithm in instructions. Indicate full posterior distribution: - Narrow: Marginal distribution of X. - Broad: Joint distribution of X and Y. Figure A1: Timeline of updating task in laboratory experiment. B.2 Consistency of Attention Across Tasks In this section I examine how consistently inattentive or consistently Bayesian subjects behave across tasks. Figure 2 in the main text includes five beliefs per subjects. But does each subject exhibit a stable level of attention? Figure B.2 shows kernel density estimates of subject-level mean inattention. While there is a strong clustering of subjects in the Broad condition who always form beliefs that are close to an implied inattention of zero, there are no such two clusterings in the Narrow treatment one at each end of the attention spectrum as could be expected from Figure 2. Instead, there is a smaller clustering at mean inattention values of between.8 and 1. Indeed, I find that many subjects in Narrow condition formed close to Bayesian beliefs in some tasks, and close to fully inattentive beliefs in other tasks. In fact, 15.5% of subjects in Narrow indicated both a fully Bayesian and a fully inattentive belief at least once. This may suggest that a subject s degree of attention to Y varies across situations to some extent, even for largely identical updating contexts. 5

56 4 3 Density Estimated subject-level mean of inattention Condition Narrow Condition Broad Figure A2: Subject-level mean of inattention to Y. N=144. For each subject I calculate the mean inattention in the five baseline tasks. The curves show kernel density estimates for each treatment (both N=72). A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention. Epanechnikov kernel with bandwidth.1. 6

57 C Robustness Treatments C.1 Task Variations Table A3: Overview of robustness tasks Task Sample space X Sample space Y Signal type Signal value Correlated data (r=.7) {95, 96,..., 14, 15} { 15, 14,..., 14, 15} (X + Y) 2 14 Larger sample space (> 1) {19, 191,..., 29, 21} {18, 181,..., 219, 22} (X + Y) 2 28 Discrete normally distributed numbers {17, 18,... 22, 23} { 5, 4,... 4, 5} X + Y 22 Signal out of X range {24, 241,..., 259, 26} { 15, 14,..., 14, 15} X + Y 23 Notes: This table provides an overview of the four robustness belief tasks. The distributions of X and Y as well as the signal structure are identical in both treatment conditions. X and Y were independently drawn from two discrete uniform distribution, i.e., every indicated outcome was equally likely. Table A4: Median inattention in robustness tasks Task Median inattention θ Mdn Mann-Whitney U test Narrow N=72 Broad N=72 (p-value) Correlated data (r=.7).59. <.1 Larger sample space (> 1) <.1 Discrete normally distributed numbers <.1 Signal out of X range <.1 Notes: This table displays group medians of implied inattention parameters by treatment condition for four additional belief formation tasks. Inattention is calculated as θ = H B H B +H N, where H B and H N denote the Hellinger distance of the subjective distribution to the Bayesian posterior and the inattentive posterior distribution, respectively. Task order was randomized within each of the two blocks. 72 subjects participated in each condition. 7

58 (robust_correlated_data) (robust_continuous) Distribution of beliefs (robust_normal_data) (robust_out_of_range) Condition Narrow Condition Broad Bayesian posterior 8 Figure A3: Distribution of elicited belief distributions about X in each one of four robustness tasks. N=72 for each condition in each task. The horizontal axis shows possible outcomes of X. The Bayesian posterior belief is provided for reference. The observed signal is indicated by the vertical dashed line. The conrresponding task configurations are shown in Table A3.

59 C.2 Face Value Heuristic and Anchoring In treatment Computation, a simple algebraic computation was added on top of the signal structure. The resulting signals provided in the five baseline tasks were average of X and Y minus (3 5) plus 35, sum of X and Y plus (2 1) minus 3, sum of X and Y plus 4 minus (4 5), average of X and Y minus (8 5) plus 1, and average of X and Y plus (3 5) plus 1. Note that given the simplicity of these calculations, it is possible that subjects did not have to execute these computations effortfully but the results automatically came to mind. This is suggested by research on dual processing (Thompson, 213). The computations were chosen such anchoring on the signal value remains equally plausible. If subjects apply a simple face value heuristic, they should ignore both the the computation and the variation of Y. Figure A4 shows raw beliefs in condition Computation, including the signal value and the signal value after accounting for the computation. There is limited evidence for anchoring on the signal value. Subjects do not simply take the signal at face value, but they take into account the computation and still neglect Y. 9

60 (1) (2) (3) Observed Signal Signal without computation Signal without computation Observed Signal Observed Signal Signal without computation Distribution of beliefs Distribution of beliefs Signal without computation Observed Signal (4) Observed Signal (5) Signal without computation Computation in Narrow Bayesian posterior 1 Figure A4: Distribution of elicited belief distributions about X in condition Computation. N=24 in each task. The horizontal axis shows possible outcomes of X. The Bayesian posterior belief is provided for reference. The observed signal is indicated by the solid dashed line, and the signal value after undoing the computation is shown by the dashed line. In all five tasks, X and Y follow independent discrete uniform distributions that were shown to subjects. Task order was randomized.

61 C.3 Elicitation Procedure 3 Density Implied inattention parameter Narrow Narrow with joint elicitation Broad Broad with sequential elicitation Figure A5: Subject-level mean of inattention to Y in four conditions. Based on N=216. For each subject I calculate the mean inattention in the five baseline tasks. The curves show kernel density estimates for each treatment (Narrow N=72, Broad N=72, Narrow with joint elicitation N=24, Broad with sequential elicitation N=48). A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention. Epanechnikov kernel with bandwidth.1. In treatments Narrow and Broad, prediction incentives are different, but the elicitation method also differs. In Narrow, subjects only indicate the marginal of X, while in Broad, subjects indicate the joint distribution of X and Y. To rule out that treatment effects are driven by this difference in what is elicited, I designed two additional treatments. In Narrow with joint elicitation, only X is paid for (as in Narrow) but the joint distribution is elicited exactly as in Broad. In Broad with sequential elicitation, X and Y are paid for (as in Broad) but now the subject first indicates the marginal 11

62 of X, and then indicates the marginal of Y on a separate screen. This way, the first screen (for the marginal of X) is exactly identical to Narrow. Figure A5 plots kernel density estimates of the within-subject mean of inattention in the five belief tasks for all four treatments. Mean inattention in the four treatments is.25 (Broad),.34 (Broad with sequential elicitation),.47 (Narrow with joint elicitation), and.57 (Narrow). These findings imply that the treatment effect is not an artifact of different elicitation methods. Harmonizing the elicitation procedure somewhat reduces the effect in the predicted direction, but prediction incentives as such have a unique effect. 12

63 C.4 Simplification Density Implied inattention parameter Condition Narrow Condition Broad Narrow without deciphering and time limit Figure A6: Implied inattention to Y in three conditions. Based on 1,944 stated beliefs. The curves show kernel density estimates for each treatment (Narrow N=864, Broad N=864, Simplification N=216). A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention. Epanechnikov kernel with bandwidth.1. To study the role of complexity in the experimental setting, an additional condition drastically simplifies the experimental procedure by removing the deciphering stage as well as all time constraints. In this treatment, subjects are paid to predict X as in Narrow, but they do not have to decipher the signal and have unlimited time to indicate their guess. Effectively, they are given the distributions of X and Y, and directly see the value of the signal. There is a statistically significant reduction in inattention relative to Narrow in this case (p =.). At the same time, inattention remains far higher than in Broad (p =.). Mean inattention is.57 in Narrow,.4 13

64 in Simplification, and.25 in Broad. Also, there is somewhat reduced bunching at fully inattentive and fully Bayesian beliefs. Considerable simplifications improve predictions, but do not eliminate the effect of Narrow incentives. Figure A6 plots kernel density estimates of the distribution of inattention parameters in Simplification together with Narrow and Broad for reference. C.5 The Form of Noise Neglect Table A5: Online experiment on form of noise neglect X Y Info structure I Observed info i U{75, 76,..., 125} U{9, 91,..., 11} X+Y 2 Individual draw U{75, 76,..., 125} U{ 25, 24,..., 25} X + Y Individual draw N (1, 4) N (5, 1) X+Y 2 Individual draw N (1, 4) N (1, 1) X+Y 2 Individual draw N (1, 4) N (1, 1) X + Y Individual draw U{5, 51,..., 15} U{5, 51,..., 15} X + Y 145 U{75, 76,..., 125} U{9, 91,..., 11} X+Y N (1, 4) N (1, 1) 2 X + 2 Y 412 N (1, 4) N (1, 1) 2 X + Y 266 N (1, 4) N (1, 1) X + Y 11 Notes: This table provides an overview of the ten belief tasks in the online experiment on the form of noise neglect. Note that for all normally distributed variables, the support was discretized to integers, truncated at µ 5 and µ + 5 and then the distributions were scaled such that the they have unit probability mass. Table A5 displays the ten tasks used in an online experiment on the form of noise neglect with 79 subjects recruited from Mturk. In five of those tasks, information values were drawn individually for each subject, while in the remaining tasks one information value was drawn jointly for all subjects to obtain higher power for a specific realization. 14

65 Figures A7 and A8 illustrate the corresponding results, which are also discussed in the main text in Section 3.2. In each of the tasks in A7, the solid reference line corresponds to Bayesian posteriors while the dashed line indicates reference beliefs under noise neglect. Figure A8 demonstrates that the form of noise neglect is not generally in line with people using a modified distribution of Y. To see this, the green line indicates a corresponding threshold: all belief on the opposite side of the Bayesian posterior are not compatible with any possible implied distribution of Y on the actual support of Y. At the same time, these tasks indicate that noise neglect is not easily reconciled with oversensitivity to the likelihood (or neglect of base rates), as would be in line with, e.g., diagnostic expectations (Bordalo et al., 218). Consider for example the task displayed in the upper right corner of Figure A8, where X U{5, 51,..., 15}, Y U{5, 51,..., 15}, I = X + Y and i = 145. Here, an information value of 145 indicates that a relatively small value of X, i.e., x < 1, has been drawn, and the likelihood increase is greatest for values of X below 1. However, people predominantly choose values above 1, close to

66 X = U{75,76,...,125} Y = U{9,91,...,11} I = (X+Y) / 2 X = N(1,4) Y = N(5,1) I = (X+Y) / Guess of X 1 Guess of X Observed signal Observed signal X = U{75,76,...,125} Y = U{ 25, 24,...,25} I = X+Y X = N(1,4) Y = N(1,1) I = (X+Y) / Guess of X 1 9 Guess of X Observed signal Observed signal X = N(1,4) Y = N(1,1) I = X+Y Guess of X Observed signal Figure A7: Raw beliefs in online experiment on the form of noise neglect. The solid reference line indicates the Bayesian posterior, the dashed line shows noise neglect. N = 79 in each task. Displayed are the five out of ten tasks in which the information value was individually drawn for each subject. The task order (of all ten tasks) was randomized at the individual level. 16

67 2 X = U{5,51,...,15} Y = U{5,51,...,15} I = X + Y 25 X = U{75,76,...,125} Y = U{9,91,...,11} I = X + Y Guess of X 15 1 Guess of X Observed signal: 145 Observed signal: 116 X = N(1,4) Y = N(1,1) I = 2 X + Y X = N(1,4) Y = N(1,1) I = 2 X + 2 Y 9 15 Guess of X 6 Guess of X Observed signal: 266 Observed signal: 412 X = N(1,4) Y = N(1,1) I = X + Y 3 Guess of X Observed signal: 22 Figure A8: Raw beliefs in online experiment on the form of noise neglect. The solid red reference line indicates the Bayesian posterior, the dashed red line shows noise neglect. The green line indicates a threshold. All belief on the opposite side of the Bayesian posterior are not compatible with any possible implied distribution of Y on the actual support of Y. These tasks therefore provide evidence against the idea that noise neglect is in line with people 17 using a modified distribution of Y. N = 79 in each task. Displayed are the five out of ten tasks in which all subjects observed the same information value. The task order (of all ten tasks) was randomized at the individual level.

68 D Mechanism Treatments D.1 Treatments Signal-to-Noise Ratio and Directional Bias Insert regression tables here. D.2 The Role of Effort: Manipulation of Stake Size Table A6: Inattentive inference and effort Tasks: Bonus round (variation of stakes) Conditions: Narrow and Broad Narrow Dependent variable: Response time Inattention θ (1) (2) (3) High stakes in bonus task ** (8.711) (.54) (.1) if Broad, 1 if Narrow ***.52*** (7.538) (.84) Treatment dummy * High stakes (11.795) (.113) Constant ***.11**.63*** (5.671) (.42) (.73) R # Observations Notes: OLS regressions. In the bonus round I randomly vary within each treatment whether incentives are 1 euro or 1 euros. Response time is the duration in seconds the subject spent on the belief elicitation page. Inattention is calculated as θ = H B H B +H N, where H B and H N denote the Hellinger distance of the subjective distribution to the Bayesian posterior and the inattentive posterior distribution, respectively. Robust standard errors clustered at participant level in parentheses. *p <.1, **p <.5, ***p <.1. 18

69 D.3 Hint Treatment 5 4 Density Implied inattention parameter Condition Narrow Condition Broad Condition Hint Figure A9: Implied inattention to Y in three conditions. Based on 95 stated beliefs. The curves show kernel density estimates for each treatment (Narrow N=36, Broad N=36, Hint N=23). A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention to noise. Epanechnikov kernel with bandwidth.5. 19

70 D.4 Confidence Ratings After finishing the baseline, robustness and bonus belief tasks in the laboratory, each of the tasks was again presented successively including all previously shown information as well as the subject s stated guess. In a list with fixed monetary amounts from euros to 6.25 euros in steps of.25 euros, subjects then indicated whether they prefer to be paid out for their stated belief, or receive this fixed amount, in case this belief task would be selected to count. Single switching was enforced. Figure A1 shows that implied inattention of the belief and stated valuations for the belief are virtually unrelated. 6 5 Switching point Inattention parameter Condition Narrow Linear fit Narrow Condition Broad Linear fit Broad Figure A1: Scatterplot and linear regression fits for valuations of stated beliefs and implied inattention by condition. Based on N=36 each for condition Narrow and condition Broad. 2

71 D.5 Switch-Role Tasks As the last part of the main baseline experiment, i.e., following the confidence tasks, subjects were (unexpectedly) presented with two additional tasks in which roles were switched with the respective other condition. The switch-role task configurations were comparable to those of the baseline tasks. Figure A11 displays group means of inattention for each of the blocks of tasks by condition. Having previously predicted X and Y in condition Broad makes subjects somewhat less inattentive than in the Narrow baseline, but not by much. A highly significant reverse treatment effect persists in teh switch-role tasks Inattention (mean ± s.e.m.) Baseline tasks Switch-role tasks Task block Condition Narrow Condition Broad Figure A11: Group means of inattention by task block and condition. Based on N=36 baseline beliefs and N=144 switch-role beliefs each for condition Narrow and condition Broad. 21

72 E Learning Treatments In the first baseline round, i.e., before receiving feedback for the first time, inattention scores do not significantly differ between the four learning treatments, as expected. Feedback Computation with Feedback Computational Feedback Imperfect Feedback Inattention (mean ± s.e.m.) Figure A12: Treatment means of inattention to Y in the first round. Displayed are implied inattention scores in the initial baseline round. Subjects have not previously received feedback when stating these guesses. Sample sizes are N = 48 in both Feedback N = 24 each in all other three conditions. 22

73 E.1 Feedback From the initial experiments we know that the neglect of Y is typically confident and occurs outside subjects awareness. The key hypothesis motivating the feedback treatments is that people fail to reflect on steps of their solution strategy that are not available to introspection or recall, interfering with learning even in the presence of surprising feedback. Condition Feedback is akin to Narrow, but also shows the actually drawn number of X after guessing it. Relative to the no-feedback benchmark (condition Narrow), there is marginally significant learning after receiving feedback for the first time (p=.6, in a regression of inattention in the second round on a treatment dummy and including task-fixed effects). After having received feedback four times, mean inattention is.27 as compared to.69 in the no-feedback baseline. Despite this sizable improvement, inattention is still significantly greater than in the fifth round of the no-feedback setting with Broad incentives (mean inattention.1, p=.). Figure A13 shows a histogram of inattention parameters, and Figure A14 histograms of the raw beliefs in condition Feedback. 23

74 .25.2 Fraction of beliefs Implied inattention parameter Figure A13: Histogram of implied inattention to Y in condition Feedback. Based on 216 stated beliefs. A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention. 24

75 Distribution of beliefs (1) (2) (3) Distribution of beliefs (4) (5) Bayesian posterior 25 Figure A14: Distribution of elicited belief distributions about X in each one of five baseline tasks of condition Feedback. N=24 for each condition in each task. The horizontal axis shows possible outcomes of X. The Bayesian posterior belief is provided for reference. The observed signal is indicated by the vertical dashed line. Task order was randomized.

76 E.2 Computation with Feedback To directly test the hypothesis that people fail to reflect on the non-accessible elements of their solution strategy, Computation with Feedback provides feedback that is identical to Feedback, but the initial signal about X and Y is modified by a simple algebraic computation. This condition is identical to the anchoring treatment Computation, but including the feedback stage. As found in the Computation condition and confirmed here, the additional computation is inconsequential for the guesses about X that subjects submit (see also Figure A12). Virtually every subject correctly accounts for the computation but then tends to forget about Y. Presented with surprising feedback about the actually drawn number, however, subjects might now first remember the conscious part of their inference strategy, i.e., undoing the calculations. The computations provide them with "a place to hang their coat" in the sense of an obvious albeit unlikely source of error. This is what I find: Adding the computation virtually eliminates learning. Figure A15 shows a histogram of inattention parameters, and Figure A16 histograms of the raw beliefs in condition Computation with Feedback. 26

77 .3 Fraction of beliefs Implied inattention parameter Figure A15: Histogram of implied inattention to Y in condition Computation with Feedback. Based on 216 stated beliefs. A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention. 27

78 Distribution of beliefs (1) (2) (3) Distribution of beliefs (4) (5) Bayesian posterior 28 Figure A16: Distribution of elicited belief distributions about X in each one of five baseline tasks of condition Computation with Feedback. N=24 for each condition in each task. The horizontal axis shows possible outcomes of X. The Bayesian posterior belief is provided for reference. The observed signal is indicated by the vertical dashed line. Task order was randomized.

79 E.3 Computational Feedback Reduced learning when algebra is added could result from increased complexity. In condition Computation Feedback, therefore, subjects have narrow incentives and receive a signal on X and Y without additional computations, i.e., the mean or sum as before. This time however, the same computations as in Computation with Feedback are added at the feedback stage. That means, instead of seeing the true value of X, subjects see a different value on which they first perform the computations and then arrive at the true value of X. The results suggest it is not computational complexity of a problem per se that reduces learning form feedback. Instead, it is precisely the consciously accessible steps of reasoning performed when doing inference that interfere with reflecting on the role of Y. Figure A17 shows a histogram of inattention parameters, and Figure A18 histograms of the raw beliefs in condition Computational Feedback. 29

80 .3 Fraction of beliefs Implied inattention parameter Figure A17: Histogram of implied inattention to Y in condition Computational Feedback. Based on 216 stated beliefs. A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention. 3

81 Distribution of beliefs (1) (2) (3) Distribution of beliefs (4) (5) Bayesian posterior 31 Figure A18: Distribution of elicited belief distributions about X in each one of five baseline tasks of condition Computational Feedback. N=24 for each condition in each task. The horizontal axis shows possible outcomes of X. The Bayesian posterior belief is provided for reference. The observed signal is indicated by the vertical dashed line. Task order was randomized.

82 E.4 Imperfect Feedback Learning in practice is often based on imprecise signals. The possibility that observed feedback is not exactly right might provide another obvious way for subjects to explain a conflict between their stated belief and received feedback, reducing learning. In an additional treatment, feedback about the true X was only correct with 8% probability, and the remaining 2% subjects would see a value of X which is not the true one. Pooling beliefs following the first four rounds of feedback, there is only a small and marginally significant positive effect of receiving this feedback on inattention relative to receiving no feedback at all (p =.9). As predicted, simple solutions for why beliefs conflict with the feedback compromise the ability to reflect on the role of Y. Figure A19 shows a histogram of inattention parameters, and Figure A2 histograms of the raw beliefs in condition Imperfect Feedback. 32

83 .4.3 Fraction of beliefs Implied inattention parameter Figure A19: Histogram of implied inattention to Y in condition Imperfect Feedback. Based on 216 stated beliefs. A parameter of θ = is consistent with Bayesian updating. θ = 1 means complete inattention. 33

84 Distribution of beliefs (1) (2) (3) Distribution of beliefs (4) (5) Bayesian posterior 34 Figure A2: Distribution of elicited belief distributions about X in each one of five baseline tasks of condition Imperfect Feedback. N=24 for each condition in each task. The horizontal axis shows possible outcomes of X. The Bayesian posterior belief is provided for reference. The observed signal is indicated by the vertical dashed line. Task order was randomized.

85 F Experimental Instructions F.1 Main Instructions in Narrow and Broad All instructions were computerized. Translated from German into English. Welcome. For your participation you will receive a fixed payment of 1. e, which will be paid to you in cash at the end. In this study you will take decisions on the computer. Depending on how you decide you can earn additional money. During the study it is not allowed to communicate with other participants. Note also that the curtain of your cubicle must be closed during the entire study. Please turn off your mobile phone now, so that other participants will not be disturbed. Please only use the designated functions on the computer and make your entries using the keyboard and the mouse. If you have questions, please make a hand signal. Your question will be answered at your seat. To proceed click "Next". Your Task You will successively receive 9 different guessing tasks. The guessing tasks are about guessing numbers that are randomly drawn. The better your guess, the more money you can earn. In each guessing task there is a random number X. The computer randomly picks X from a range of possible numbers. You will receive an encrypted hint about which number was actually drawn, and you can then indicate your guess about X. There are 9 rounds in total. In each round you receive a new guessing task. That means, in each round the computer again determines a number X independently of the other rounds. Your payoff depends on how precisely you guess, that means how accurate your guess is. At the end of the study, one of the 9 rounds is picked at random and you will be paid according to the precision of your guess in that round. 35

86 The Guessing Tasks Example. Imagine there are exactly 3 balls. These 3 balls have the following numbers on them: 1, 2, 3. In this example, the number X is determined as follows: The computer randomly draws one of these three balls. Each ball is drawn with equal probability. It is equally likely that the 1 will be drawn, that the 2 will be drawn, or that the 3 will be drawn. The number X is then the number of the ball that was randomly drawn by the computer. However, you will not be told which number X was drawn. Instead you receive an additional hint. You can look at this hint, before you guess the number X. Please note: For each guessing you will be informed about which numbers can be drawn. In different guessing tasks, different numbers can be drawn. Sometimes the numbers repeatedly occur across rounds. However, the draws in these rounds are completely independent of one another. The additional hint can give you different types of information in different rounds. In each round you will learn anew, what the additional hint means. Therefore you should pay attention in every new guessing task to which information the hint indicates. Your guess. You can state your guess by allocating 1 percentage points to the different numbers. The more certain you are, that a particular number was drawn, the more points you should allocated to this numbers. Similarly, the more certain you are, that a particular was not drawn, the fewer points you should allocate to this numbers. The sum of your allocated points must be exactly 1. In the example above, if after receiving the additional note you are, for example, sure that X = 3, then you should allocate 1 points to the number 3, and points to both the numbers 1 and 2. In the example above, if after receiving the additional note you are, for example, sure that X = 2, then you should allocate 1 points to the number 2, and points to both the numbers 1 and 3. In the example above, if after receiving the additional note you think, for example, that the number 3 have definitely not been drawn, 36

87 but the 1 and 2 have been drawn with equal probability, then you should allocate 5 points each to the number 1 and 2, and points to the number 3. You can arbitrarily allocate the points. However you can only allocate full points, that means for example that you cannot allocate half points. For instance, you could allocate 21 points to number 3, 47 points to number 2, and 32 points to number 1. The more points you assign to the number that was actually drawn, the more money you can earn. Similarly, the fewer points you allocate to those numbers, that are not equal to X, the more money you can earn. The calculation of your payoff will be explained in greater detail in the following section. Your payment In addition to your show up fee you will be paid based on how precisely you guessed. To this end one of the 9 rounds will randomly be picked and you will be paid according to the precision of your guess in that round. This means for you that each one of your guesses is potentially relevant for your payment and accordingly you should carefully think through every guess. You can either earn and additional 1 e or e from your guessing task. While the following explanation might look difficult, the basic principle is very simple: the better your guess, that is the more percentage points your guess assigns to the actually drawn number and the fewer percentage points it allocates to every wrong number, the more likely it is that you receive the 1e. Concretely this means the following: In expectation you will earn most money if you allocate your points according to how probable you find it that the respective numbers was drawn (with 1 point = 1 percent). If you have understood this, it is not necessary for the maximization of your earnings to read the following section on the details of the calculation of your additional payment. You can then directly click on Next. For your information: Details on the calculation of your additional earnings. For working on the guessing tasks it is not necessary that you read and fully understand 37

88 the following section on the calculation of your payoff. you can also skip this part. After you have stated your guess, the computer will randomly draw another number kj This number is between and 2,. (More precisely, this numbers is drawn from a discrete uniform distribution on the interval from to 2,.) You will then receive the 1 e if the sum S is smaller or equal to k. S is the sum of the following elements: The squared deviation between the number of points that you allocated to the actually drawn numbers X, and 1 points. For each possible number, that has not been drawn (i.e., every other number than X): The squared deviation between points and the number of points that you allocated to this numbers. An exact mathematical formula of the sum S is displayed in the footnote. 34 If the sum S is bigger than k you will receive e. Accordingly, the payoff rule is as follows: Payment = 1. e, if S k Payment =. e, if S > k This means the following: If the sum of the squared deviations exceeds a particular value k, you will receive e. If, however, the sum of the squared deviations is smaller than k, you will receive 1 e in addition. You can notice here that it should be your goal a) to keep the difference between the points allocated to X and 1 points as low as possible, that is to allocate as many points as possible to X, and b) to allocate as few points as possible to ever other number than X. An example: Let us assume that the computer has randomly drawn the number X = 3, while the numbers 1, 2 and 3 could have been drawn with equal probability. Also 34 Footnote text: Exact mathematical formulation: There are N possible number from which X is drawn. In the example, N = 3. The number of points that you allocate to the ith of the N numbers is p i. The indicator function 1 i takes the value 1, if X is the ith number, and otherwise. The sum S is calculated as follows: S = N i=1 (1 i p i ) 2. The expected payoff amount is maximized by indicating the probability distribution of the numbers after receiving the additional hint. 38

89 the number k = 5, For the following guesses you would receive the indicated payments. In particular this means the following: If you allocate all 1 points to the right number X, you will received the 1 e in any case. However, you will also receive 1 e in many cases in which you allocate less than 1 points to X. The more points you allocate to the right number X, the more likely it is, that you receive the 1 e. In expectation, you will earn the most money if you allocate the points according to how probable you think it is that the respective number was drawn. Please note: It is not necessary, to allocate 1 points to the number that you think is most likely. As you can see in the examples of the table, you can also win 1 e if you have allocated less than 1 points to the right number X. Your earnings depend on the randomly drawn number k. Your guess in one randomly picked round will be paid. The guessing task that is payoff relevant for you is determined by the computer at the end of the 39

90 study. Therefore you should indicate your best guess in each guessing task, independent of all other guessing tasks. Summary In each round it is your task to state a guess about the number that was randomly drawn by the computer. Before this, you will get a computer-generated, encrypted hint. For each guessing task you will see this additional hint and you can subsequently indicate your guess. Which hint you will receive, and how this hint is encrypted will be explained in the following. For the deciphering of the hint and your subsequent guess there is a time limit. You will previously be informed about how much time you have. The remaining time will be displayed while working on the tasks. Encryption of Hints You receive additional hints that have been encrypted by an encryption device. The encryption device transforms each hint (a number) into a letter code. You first need to decrypt the letter code back into a number in order to use the hint. Decryption of the additional hint. When you get an encrypted sequence of letters as hint, you can decipher this hint by following these steps: a. Transform the sequence of letters into a number using the code table. b. Add 2 to the number Before every guessing task you will receive an encrypted hint that you can decipher before stating your own guess. Whenever you receive a hint, you will see the code 4

91 table as well as the decryption instructions. That means you don not have to remember the decryption procedure. You will soon get the opportunity to practice the decryption on an example hint. Control Questions Please notify one of the experiments now if you have questions about the instructions so far. If there is something that is unclear to you, please re-read the respective information carefully. You can return to the previous pages by clicking Back. If you click on Proceed to control questions, you will receive several control questions, which ensure your understanding of the instructions. You will not get paid for the control questions. However, you have to correctly answer all control questions to proceed to the guessing tasks. After you have correctly answered all control questions, you will be presented with the first guessing task. F.2 Control Questions Control Question 1 of 9 What is your main task in this study? There are several number from which X can be drawn. I need to add these numbers up to a sum. I guess the drawn number X. Control Question 2 of 9 The numbers from which X is drawn vary across rounds. Sometimes the numbers occur in different rounds. For example, it could be that in two different rounds, the number X is randomly drawn from the number 1, 2, and 3. Please evaluate the following statement: In both round, each of the 3 numbers is drawn with equal probability. 41

92 Wrong. If, for example, the 1 was drawn in the first round, it is more probable that 1 will not be drawn in the next round. Correct. Both rounds are completely independent. The draw in the first round has no influence on which number is drawn in the second round. Control Question 3 of 9 In guessing X, how can you make most money? By allocating the points to the numbers as precisely as possible based on how certain I am, that the respective number is X. By varying my guess and allocating by instinct sometimes more points to high numbers and sometimes more points to low numbers. Control Question 4 of 9 After you have read the description of the guessing task and received the additional hint, you think that the number 2 is the most likely drawn number among the numbers. However you are not certain that it is the 2. Assess the following statement: To maximize my payoff I have to put all 1 points on the number 1. Correct. It is only this way that I can earn the 1 euros. Wrong. While I should put more points on the 2 than on all other numbers, I should not putt all points on the 2, because I am not certain. If for example i am 6% sure that X = 2, I should put exactly 6 points on the 2, and allocate the remaining 4 points to the other numbers. This way it is most probably that I earn the 1 euros. Control Question 5 of 9 Which of your guess is payoff-relevant? 42

93 Every guess is paid out. No guess is paid out. A randomly picked guess is paid out. Control Question 6 of 9 Imagine the number X is drawn with equal probability from the following four numbers: 5, 6, 7, 8. You have no additional information. Please indicate how in this case you should allocate the 1 points to the four numbers such that you make winning the 1 euros as likely as possible. Start by picking a number in the selection box to the left and assign a number of percentage points in the input field to the right. Use further input rows if you want to assign percentage points to other numbers. Control Question 7 of 9 As before the number X is drawn with equal probability from the following four numbers: 5, 6, 7, 8. Please imagine now that after deciphering the hint you are certain that the 7 was drawn. Please indicate how in this case you should allocate the 1 points to the four numbers such that you make winning the 1 euros as likely as possible. If you want to allocate percentage points to a number then you do not have to enter this into an extra row, but you can simply skip this number ( points will automatically be allocated). Control Question 8 of 9 Imagine you receive the hint: AJ. Please decipher the hint and enter your result below. a. Transform the sequence of letters into a number using the code table. 43

94 b. Add 2 to the number The decrypted hint reads: Control Question 9 of 9 Imagine now you receive the hint: ACJ. Please decipher the hint and enter your result below. a. Transform the sequence of letters into a number using the code table. b. Add 2 to the number The decrypted hint reads: F.3 Task Instructions Next to X another number was drawn by the computer, Y. Whether a participant has to guess Y as well was randomly determined at the beginning of the study and has no impact on the size of possible earnings. [ Treatment Narrow: To you applies the following: You indicate a guess only about X and will be paid for your guess of X as described. ] [ Treatment Broad: To you applies the following: You guess both drawn numbers, X 44

95 and Y. One of the numbers will later be picked and you will be paid for your guess of this number as described. ] [ The following description varies by task ] X was randomly drawn from the following 5 numbers between 8 and 12, where each number was equally likely: 8, 9, 1, 11, 12. Y was randomly drawn from the following 7 numbers between -3 and 3, were each number was equally likely: -3, -2, -1,, 1, 2, 3. X and Y were drawn independently. [ Treatment Broad: You will guess X and Y simultaneously, that means in each entry row you have to pick both a number for X and a number for Y and indicate a percentage alongside, which is your guess that these two numbers were drawn together. ] When you click Next, you will first receive your additional hint on the following page. You have 5 minutes time to decipher the hint. Then you have another 5 minutes of time to indicate you guess. The remaining time will be displayed on the upper right corner of the pages. Your Additional Hint Your additional hint for the guess of X [ X and Y ] is: FJ. The completely decrypted hint indicates the sum of the 2 drawn numbers, i.e., X + Y. Decryption Instructions. a. Transform the sequence of letters into a number using the code table. b. Add 2 to the number [ Calculator provided. ] On the next page you will see the entry fields for your guess. You can now enter your decrypted additional hint below, then it will be displayed 45