Accounting Standards, Implementation Guidance, and Example-Based Reasoning

Transcription

1 DOI: /j X x Journal of Accounting Research Vol. 45 No. 4 September 2007 Printed in U.S.A. Accounting Standards, Implementation Guidance, and Example-Based Reasoning SHANA CLOR-PROELL AND MARK W. NELSON Received 10 November 2005; accepted 7 September 2006 ABSTRACT This paper examines interpretation of accounting standards that provide implementation guidance via affirmative or counter examples. Based on prior psychology research, we predict that practitioners engage in example-based reasoning such that they are more likely to conclude that their case qualifies for the same treatment as the example. We test our predictions in two experiments in which participants judge the appropriateness of income-statement recognition. Experiment 1 uses Masters of Business Administration (MBA) students and varies example type (affirmative, counter) and case (revenue recognition, expense recognition) in a 2 2 design. Experiment 1 supports our predictions. Experiment 2 uses more experienced practitioners, and varies example type (affirmative, counter, both) in a 1 3 design. Experiment 2 supports the use of example-based reasoning, and indicates that practitioners in the both condition respond as if they had only received an affirmative example. These results have implications for understanding how guidance that accompanies accounting standards can result in aggressive or conservative application of standards. S. C. Johnson Graduate School of Management, Cornell University. We thank Rob Bloomfield, Tom Dyckman, Merle Erickson, Tom Fields, Max Hewitt, Frank Hodge, Jane Kennedy, Bob Libby, Kathy Rupar, Steve Salterio, Nicholas Seybert, William Tayler, an anonymous reviewer, and workshop participants at Cornell University and the University of Washington for helpful comments. We also thank Cornell s Johnson Graduate School of Management for financial support. 699 Copyright C, University of Chicago on behalf of the Institute of Professional Accounting, 2007

2 700 S. CLOR-PROELL AND M. W. NELSON 1. Introduction Accounting standards in the United States currently consist of a mix of principles and rules (SEC [2003]), and the Financial Accounting Standards Board (FASB [2002]), International Accounting Standards Board (IASB [2002]), and Securities and Exchange Commission (SEC [2003]) have recently advocated moving to a more principles-based financial-reporting system that avoids bright-line rules and requires more professional judgment to determine appropriate accounting. Much research suggests that practitioners sometimes employ the latitude inherent in standards to make choices consistent with their incentives, such that regulators need to strike the right balance of incentives to encourage accurate reporting when latitude exists (Nelson [2003]). We examine whether, even when practitioners intend to report accurately, the process by which they deal with latitude might produce inaccurate reporting. We focus on practitioners use of examples when determining appropriate application of standards. In practice, examples can be provided by standard setters as implementation guidance when the standard is issued (e. g., Statement of Financial Accounting Standards (SFAS) No. 106 (FASB [1990]), SFAS No. 123(R) (FASB [2004]), SFAS No. 133(FASB [1998])). Examples also can be provided by standard setters subsequent to issuance (e. g., Emerging Issues Task Force (EITF) 94-3 (FASB [1994]) and EITF (FASB [1999])), by regulators via enforcement actions (e. g., Accounting and Auditing Enforcement Release (AAER) 108 (SEC [1986])), and by the business press. We use the term example-based reasoning to refer to the psychological process by which practitioners apply such examples. Prior psychology research examining similarity judgments (e. g., Tversky [1977]) and priming (e. g., Higgins, Bargh, and Lombardi [1985]) suggests that practitioners may conclude that the treatment illustrated by the example is more appropriate for their situation than is actually justified by the underlying facts. Thus, a practitioner might tend to conclude that a particular accounting treatment is appropriate for their case when presented with implementation guidance that provides an example of acceptable reporting (which we call an affirmative example ), and for the same case might tend to conclude that a particular accounting treatment is not appropriate for their situation when presented with implementation guidance that provides an example of unacceptable reporting (which we call a counter example ). Two features of the accounting setting make it difficult to infer that results from prior psychology research generalize, and also present design challenges for our experiments. First, unlike the generic contexts used in prior psychology research, in which examples are abstract and not intended to communicate decision thresholds, a unique feature of the accounting context is that standard setters may choose specific examples to communicate thresholds or conditions that they deem necessary for

3 EXAMPLE-BASED REASONING 701 a treatment to be appropriate. For instance, an affirmative example in which two facts favor revenue recognition and two do not could communicate to participants that at least 50% of case facts must favor revenue recognition for recognition to be appropriate. Results that a participant is more likely to recognize revenue when presented with that example could be attributed either to the psychological factors we hypothesize or to the participant s case meeting the threshold communicated by the example. To discriminate between psychological and threshold-based explanations, our experiments present participants with affirmative examples in which all facts favor recognition (thus setting a very high threshold for recognition that discourages concluding that recognition is acceptable), and/or with counter examples in which all characteristics favor nonrecognition (thus setting a very low threshold for recognition that encourages concluding that recognition is acceptable). In this design, the psychological and thresholdbased explanations yield opposite results, because recognition is more likely for affirmative examples than counter examples under the psychological explanation, but less likely under the threshold-based explanation. As a consequence, this design biases away from finding support for the psychological effects that we predict, and in fact requires that the effect of example-based reasoning be strong enough to offset any threshold-based reasoning that occurs. A second distinctive feature of the accounting context is that accounting practitioners may have incentives to report inaccurately. Three aspects of our experiments allow us to discriminate between psychological and incentive-based explanations. First, we inform participants that their goal is to report accurately, and we use comprehension checks and pre-example judgments to identify the extent to which participants internalize that goal, so we can compare responses between those who internalize the accuracy goal and those who do not. Second, we gather data both from Masters of Business Administration (MBA) students (for whom setting an accuracy goal might be relatively easier) and from experienced practitioners in accounting and finance to determine if results are robust to potential intrusion of more knowledge of financial reporting and incentives. Third, and most important, we design our experiments to insure that, while reporting incentives may affect the general levels of behavior we observe, they cannot produce the specific directional predictions that underlie our hypothesis tests. We also test for and do not find patterns of results that would be implied if participants judgments were driven by incentives to report inaccurately. We report the results of two experiments. Experiment 1 utilizes a 2 2 between-subjects design in which MBA students judge the appropriateness of income-statement recognition, varying case (revenue, expense) and example type (affirmative, counter). Participants receive a case in which they must determine how likely it is that the transaction in question will qualify for revenue or expense recognition, basing their judgment on either an

4 702 S. CLOR-PROELL AND M. W. NELSON affirmative example in which recognition was allowed or a counter example in which recognition was not allowed. Results from experiment 1 indicate that, for both the revenue- and expense-recognition cases, participants are more likely to believe recognition is appropriate when given an affirmative example than when given a counter example. From the perspective of the net income implied by participants judgments, participants are on average slightly conservative, but example type and case interact, with net income higher when revenue standards use affirmative examples and when expense standards use counter examples. Debriefing data indicate that these results were obtained when participants reported using a similarity-based or priming-based judgment process, but not when participants reported using another judgment process. Experiment 2 focuses on the revenue-recognition context, and examines whether results from experiment 1 generalize to more experienced finance and accounting practitioners. Experiment 2 also investigates whether providing practitioners with both an affirmative and a counter example can reduce the effects of example-based reasoning. Accordingly, experiment 2 utilizes a 1 3 between-subjects design in which practitioners judge the appropriateness of revenue recognition, varying example type (affirmative, counter, both). Results from experiment 2 indicate that the results of experiment 1 generalize to more experienced participants that have several years of post-mba experience in accounting and finance. Results also indicate that providing both affirmative and counter examples is not an effective method of reducing the effects obtained in experiment 1, since practitioners who receive both an affirmative and a counter example respond as if they only attend to the affirmative example. Overall, these results indicate that, even when practitioners are encouraged to report accurately, and even when examples provided in implementation guidance communicate relatively extreme thresholds, reporting judgments are influenced by the psychological processes that underlie application of implementation guidance in ways that may not be intended by standard setters. Our results also indicate that providing a balance of affirmative and counter examples may not be an effective approach to reducing example-based reasoning. Therefore, standard setters may want to consider the directional effect of example-based reasoning (conservative or aggressive) created by the combination of accounting standard and example type included in any implementation guidance that they provide, particularly if latitude in standards and implementation guidance increases with a shift towards a more principles-based regime. The next section provides the background information and develops our predictions. The third and fourth sections describe the experiments used to test our predictions and present the results of each experiment. Section 5 discusses the implications.

5 EXAMPLE-BASED REASONING Background and Hypotheses Prior accounting research has investigated various aspects of auditors use of examples. 1 Salterio [1996] provides evidence that, when presented with a set of precedents that uniformly indicated a particular treatment, auditors are more likely to rely on precedents that they perceive to be similar to their case, regardless of source of precedent or client preference. Extending this work, Salterio and Koonce [1997] provide evidence that, when presented with a set of conflicting precedents, auditors are more likely to rely on precedents that favor their client s preferred treatment, similar to other research indicating significant effects of incentives when standards do not clearly specify a particular reporting alternative (e. g., Hackenbrack and Nelson [1996], Kadous, Kennedy, and Peecher [2003], Nelson, Elliott, and Tarpley [2002]). In contrast to these studies, our focus is not on the extent to which reporting preferences affect practitioners reliance on examples. 2 Rather, our focus is on determining whether practitioners use of examples might be affected by the psychological processes underlying example-based reasoning, holding constant that practitioners intend to report accurately. Specifically, we investigate whether practitioners who apply implementation guidance may tend to believe that an example included in that guidance is more relevant to their own situation than actually may be the case. Two streams of psychology research underlie our hypotheses. First, an extensive literature in psychology examines similarity comparisons and classification judgments. A simple model developed by Tversky [1977] provides a useful framework. In that model, the similarity between two items, A and B, isafunction of the features that A and B share, less the features that A and B do not share. Prior research suggests that the weight associated with shared features is often overstated, and the weight associated with unshared features is often understated when unshared features are diagnostic (Frederick and Libby [1986], Smith and Kida [1991], Church [1991], Bamber, Ramsay, and Tubbs [1997]) but overstated when they are not (Nisbett, Zukier, and Lemley [1981], Hackenbrack [1992], Glover [1997], Hoffman and Patton [1997], Shelton [1999]). Assuming the features of an example are all relevant, prior research thus suggests that the similarity between the 1 A related literature examines analogical reasoning, whereby decision makers solve problems by identifying an analog that is structurally similar to their decision problem and mapping elements of the analog to their own problem (Novick [1988], Holyoak and Thagard [1997]). Prior accounting research has examined this type of analogical reasoning in tax, auditing, and managerial accounting settings (Marchant [1989], Marchant et al. [1991, 1993], Matsumura and Vera-Munoz [2005]), providing evidence about the circumstances in which accountants are most likely to identify and apply solutions implied by an analog. 2 We use the term example rather than precedent because implementation guidance may also consist of examples that describe hypothetical transactions for the purpose of providing implementation guidance.

6 704 S. CLOR-PROELL AND M. W. NELSON example and the transaction under consideration is likely to be overstated, which could encourage practitioners to conclude that the treatment indicated by the example is appropriate for their transaction. A second relevant psychology literature involves priming, whereby some stimulus activates associations in memory that affect subsequent processing of information. Prior research on priming in the attitude and impressionformation literatures has found that the priming of an applicable construct, such as an evaluative tone, can increase the likelihood that the construct is used to process a subsequent stimulus (Higgins, Bargh, and Lombardi [1985], Levin, Schneider, and Gaeth [1998]). If an example establishes such an evaluative tone, an example for which recognition was allowed (not allowed) could lead practitioners to conclude that there is a higher likelihood of recognition (nonrecognition) for the transaction under consideration, regardless of the specific facts of the transaction. Key for our purposes is that both of the example-based processes described above (similarity assessment and priming) offer the same directional prediction about practitioners applying implementation guidance that includes examples: practitioners believe the treatment used in an example is more appropriate for their situation than is actually the case, because they overstate the similarity between their case and the example and/or because the example primes them to focus on the outcome indicated in the example. If a case is being compared with an affirmative example, practitioners are more likely to conclude that the accounting treatment used in the example is allowable for the case. In contrast, if the case is being compared with a counter example, practitioners are more likely to conclude that the accounting treatment is not allowable for the case. Thus, example-based reasoning implies a main effect of example type (affirmative/counter) in judged appropriateness of accounting treatment: H1: Practitioners provided with an affirmative (counter) example are more (less) likely to conclude that an accounting treatment is appropriate. The practical effect on net income of support for H1 depends on whether the accounting treatment in question involves revenue or expense recognition. Income is higher when revenues are recognized and lower when expenses are recognized. Therefore, if H1 is supported, lower income should result from providing counter examples to guide revenue-recognition decisions and affirmative examples to guide expense-recognition decisions, since those are the circumstances that discourage revenue recognition and encourage expense recognition. Likewise, support for H1 implies that higher income should result from providing affirmative examples to guide revenue-recognition decisions and counter examples to guide expenserecognition decisions. This reasoning suggests that H1 can be restated as an interaction between example type (affirmative/counter) and case (revenue/expense) in predicting the income resulting from example-based reasoning:

7 EXAMPLE-BASED REASONING 705 H2: The net income implied by practitioners reporting judgments for a revenue (expense) case is higher after receiving an affirmative (counter) example than after receiving a counter (affirmative) example. The example-based reasoning perspective also suggests that more accurate reporting might be encouraged by providing an equal mix of both affirmative and counter examples. If practitioners tend to overweight shared features when making similarity comparisons, overstated similarity assessments between the practitioner s case and an affirmative example could be counterbalanced by overstated similarity assessments between the practitioner s case and a counter example. If practitioners are influenced by priming, the positive prime provided by the affirmative example is counterbalanced by the negative prime provided by the counter example. 3 Therefore, in addition to replicating the results of experiment 1 with more experienced practitioners, experiment 2 tests the following hypothesis concerning the use of both affirmative and counter examples in a revenue case: H3: Practitioners provided with an affirmative (counter) example are more (less) likely to conclude that an accounting treatment is appropriate than are practitioners provided with both an affirmative and a counter example. 3. Experiment METHOD Design Overview. Experiment 1 employs a 2 2 between-subjects design, crossing two example conditions (Affirmative/Counter) and two case conditions (Revenue/Expense). Affirmative examples describe the facts of an accounting transaction that is likely to qualify for revenue/expense recognition. Counter examples describe the facts of an accounting transaction that is not likely to qualify for revenue/expense recognition. The revenue case requires a recognition determination for a bill-and-hold sale. The expense recognition case requires a technological feasibility determination for a computer software product Participants. One hundred twenty-five MBA students enrolled in an MBA-level intermediate-accounting course participated voluntarily in this study. Using MBA students as participants allowed us to enhance experimental power by reducing intrusion from experienced practitioners reporting incentives and their knowledge of existing specific standards and precedents. Data collection occurred at the start of the course (January, 3 Salterio and Koonce [1997] provide evidence that auditors provided with mixed precedents attend more to the precedents supporting the reporting position favored by their client. We focus on circumstances where a practitioner s preference is accurate application of the standard (rather than attaining a favored reporting position).

8 706 S. CLOR-PROELL AND M. W. NELSON 2005), before coverage of bill-and-hold sales or software capitalization, to ensure that participants were familiar with financial accounting and basic revenue- and expense-recognition standards but lacked knowledge of the specific accounting guidance relevant to our experiment. Experiment 1 participants have an average of five years of work experience and are an average of 29 years old. Seventy-three percent are male. The students all self-selected to take at least one elective financialaccounting course that prepares them for financial-reporting or financialanalysis responsibilities Manipulation of Case. The purpose of the case manipulation is to vary the direction of the net-income implications of participants judgments, with recognition resulting in higher net income for the revenue case and lower net income for the expense case. We choose to examine bill-and-hold sales and software development cases because extant accounting guidance (AAER 108 (SEC [1986]) and Staff Accounting Bulletin (SAB) 101 (SEC [1999]) for bill-and-hold sales; SFAS No. 86 (FASB [1985]) for software) indicates multiple characteristics that affect the likelihood that recognition is appropriate. In both cases, participants determine the probability that a transaction by Capital Auto Parts, Inc. (CAP) qualifies for revenue (or expense) recognition. Participants given the revenue case receive facts relating to a recent bill-and-hold sale, and must consider whether the seller (CAP) should recognize (versus defer) revenue for a product that the buyer has purchased for delivery in the following period. Participants given the expense case receive facts relating to CAP s recently developed computer software product, and must consider whether CAP should recognize as expense (versus capitalize) software-development costs (according to SFAS No. 86 (FASB [1985]), software costs can be capitalized after technological feasibility has been reached). While the revenue and expense cases may differ in a number of respects that influence participants assessments of the probability that recognition is appropriate, the important difference for our purposes is the directional effect of case on net income revenue recognition increases income while expense recognition decreases net income. Each case provides four key facts relating to the transaction in question. Two facts support concluding that recognition is appropriate and another two facts support concluding that recognition is inappropriate, so it is unclear from only the case materials whether recognition is appropriate. Facts provided in the revenue case are adapted from AAER 108 (SEC [1986]); facts provided in the expense case are adapted from SFAS No. 86 (FASB [1985]). The order of facts and whether a given fact supports recognition is balanced between subjects. 4 Pilot tests confirmed that the facts are similar 4 Specifically, to balance any order effects, facts presented in each case appear in 16 different orders. The orders are selected so that each fact appears in each position an equal number of times. Each fact is either in favor of or against recognition an equal number of times. Each

9 EXAMPLE-BASED REASONING 707 in terms of the extent to which they provide important information that does or does not support recognition. Each case is also accompanied by a statement of basic revenue- or expenserecognition principles (adapted from Kieso, Weygandt, and Warfield [2004]). The statement was designed to provide participants with a simplified version of an accounting standard that contains some latitude. This statement was identified as the relevant standard, but participants were also informed that various aspects affect the likelihood that recognition is appropriate, and that no individual aspect guarantees that recognition is appropriate Manipulation of Example. An important feature of the accounting setting is that standard setters provide examples as implementation guidance to communicate information to practitioners. Consequently, any example provided in a standard could be used by a practitioner in at least two ways: It could provide the basis for an example-based comparison, or it could impart information about some decision threshold. For instance, if an affirmative example contains two positive facts (consistent with recognition) and two negative facts (inconsistent with recognition), it can be viewed as indicating the decision threshold of two elements are sufficient for recognition to be allowed. Practitioners applying that example to a mixed-fact case could conclude that recognition is allowed, either because they are applying example-based reasoning or because the case satisfies the decision threshold communicated by the example. As our focus is on example-based reasoning, we wanted to use examples that only support our hypothesis when an example-based process is used, and not when a decision threshold is being applied. Therefore, the affirmative (counter) example used in the experiment describes a situation in which all four of the key facts relating to the transaction support recognition (nonrecognition). The decision threshold that can be extrapolated from these examples is recognition is likely to be allowed (disallowed) when all four elements are present (absent), which biases against concluding that recognition is (is not) justified for a case that has a mix of positive and negative elements. Thus, to the extent the examples communicate a decision threshold, the threshold biases away from our hypothesis, because the affirmative example sets a high threshold for recognition (all four features present) and therefore discourages recognition, while the counter example sets a low threshold for recognition (all four features not present) and therefore encourages recognition. However, to the extent participants pursue example-based reasoning as we hypothesize, they are more likely to fact is not always followed or preceded by the same other fact. The orders are selected so that each fact is sometimes consistent with and sometimes in opposition to each other fact in the case. Finally, the orders are selected so that facts indicating recognition are always followed by facts indicating nonrecognition and vice versa. Two recognition facts never appear next to each other and two nonrecognition facts never appear next to each other.

10 708 S. CLOR-PROELL AND M. W. NELSON conclude recognition is justified in the affirmative example case than in the counter example case Materials and Procedure. Appendices 1 and 2 include task and case information, the standard, the affirmative example, and the counter example used in the revenue and expense treatments, respectively. Participants first read a case that considers the appropriate accounting treatment for either a bill-and-hold sale (Revenue case) or costs related to the development of a computer software product (Expense case). Each case contains background financial information and the specific facts of the transaction in question. The background financial information is based on the CAP case used by Nelson, Smith, and Palmrose [2005] and adapted from Braun [2001] and Libby and Kinney [2000]. The background information is intended to indicate to participants that the treatment of the transaction in question has a relatively large financial impact on the profitability of the company. Specifically, recognizing revenue or expense from the transaction in question results in a change in net income that is almost 5% of the prior year s net income and 0.5% of the prior year s total assets. Participants are provided a case and a relevant standard (but not an example that provides implementation guidance). Participants are informed that they have an incentive to report net income accurately rather than conservatively or aggressively, and asked to indicate their incentive to reinforce it and provide data for a comprehension check. Participants also are informed that, given the case materials and standard, they should think that the likelihood of qualifying for recognition is 50%. Participants are then asked to indicate on a 100-point scale the probability that the transaction in the case qualifies for revenue or expense recognition. The lower (upper) end of the scale is marked numerically as 0 (100) and the description associated with the number is definitely does NOT qualify ( definitely qualifies ). This judgment serves as a pretest baseline to assess the effect of examples, and also provides evidence about whether participants internalized instructions that conveyed an accuracy goal and a 50% pre-example likelihood of the case qualifying for recognition. 6 Next, participants are asked to assume that the standard is accompanied by an example that is intended to provide further guidance. The example is 5 The affirmative and counter examples are consistent with each other, because the affirmative example indicates that all positive attributes allow recognition and the counter example indicates that all negative attributes do not allow recognition. Thus, the processes that we investigate are distinct from the effects of counter-factual reasoning as investigated by Heiman [1990], Koonce [1992], and Kadous, Krische, and Sedor [2005], in which considering inconsistent evidence debiases optimism or overconfidence. 6 By encouraging participants to hold a prior belief around 50% and then eliciting this prior belief, our design enables us to decrease the amount of noise in the pretest judgment while still allowing participants to set their own prior, which has been found to play an important role in prior belief-revision research (McMillan and White [1993]). It also minimizes the risk of ceiling or floor effects.

11 EXAMPLE-BASED REASONING 709 either an affirmative example or counter example as dictated by the treatment condition. After reviewing this information, participants are again asked to indicate the probability that the transaction in the CAP case qualifies for recognition as revenue or as expense, depending on the treatment condition. The experiment concludes with participants completing a short debriefing questionnaire. 3.2 RESULTS Comprehension Checks. The case materials indicate to participants that they should have an incentive to report net income accurately (i.e., unbiased) rather than conservatively (i.e., understated) or aggressively (i.e., overstated). The results of a comprehension-check question reveal that 10 participants (8%) do not indicate they have an accuracy incentive. Dropping these participants from the analysis does not affect the results that follow, so all analyses include these participants. The case materials also indicate that, prior to receiving an affirmative or counter example, there is a 50% probability that the transaction qualifies for recognition. The results of the pre-example judgment provide evidence about the extent to which participants internalize the 50% prior. A majority of participants (77) select 50% as the pre-example judgment, but a significant minority (48) do not. Dropping these participants from the analysis does not affect the results that follow, so all analyses include these participants Hypothesis Tests. H1 predicts a main effect for participants judgments of the probability that the transaction in the case qualifies for recognition, with participants judgments higher when presented with an affirmative example than when presented with a counter example. Mean judgments are shown in table 1, panel A for pre-example judgments, post-example judgments, and the difference between pre- and post-example judgments. As shown in table 1, panel B, a general linear model (GLM) with post-example judgment as the dependent variable indicates a significant main effect of example type (F = 11.37, p = 0.001), but an insignificant main effect of case (F = 1.17, p = 0.194), and an insignificant interaction between example type and case (F = 1.13, p = 0.289). 8, 9 This result supports H1 by indicating that 7 All analyses are presented using the post-example judgment as the dependent variable. Results are similar if the dependent variable is the difference between the pre- and post-example judgments, and when pre-example judgment is included as a covariate. In addition, using an indicator variable set to one if the pre-example judgment is 50% and zero otherwise reveals that the covariate does not significantly interact with example type or case, indicating that results do not depend on whether participants initial judgment deviated from 50%. When the pre-example judgment is the dependent variable, neither the main effects nor the interaction are significant. 8 Order is not significant in any analyses, so it is dropped from analyses and is not discussed further. 9 All data in both experiments are analyzed parametrically since the assumptions of normality and equal variance are met. However, the same results of hypothesis tests are obtained when rank-transformed data are used to perform nonparametric analyses.

12 710 S. CLOR-PROELL AND M. W. NELSON TABLE 1 Results of Experiment 1: Judged Probability that Transaction Qualifies for Recognition Panel A: Cell means Revenue Case Expense Case Collapsed across Case Pre- Post- Pre- Post- Pre- Postexample example example example example example Example Type N Judgment Judgment Diff. N Judgment Judgment Diff. Judgment Judgment Diff. Affirmative Counter Collapsed by type Panel B: Test of H1 Sum of Mean Source df Squares Square F Value p-value Case 1 1, , Example 1 8, , Case Example Error , Participants in experiment 1 judge the probability that the transaction in the case qualifies for revenue or expense recognition. This table reports descriptive statistics and hypothesis tests about participants judgments. Participants receive a basic revenue (expense) recognition standard and apply it to a case that always contains an equal mix of facts supporting recognition and facts supporting nonrecognition. Participants make pre-example judgments (based on only the case information and general standard) and post-example judgments (after receiving either an affirmative example in which all facts favor recognition and recognition is allowed, or a counter example where all facts favor nonrecognition and recognition is not allowed), with the example serving as implementation guidance that supplements the standard.

13 EXAMPLE-BASED REASONING 711 the judged probability of recognition is higher when provided an affirmative example than when provided a counter example. Another way to consider the effects of example-based reasoning is to focus on the effect of recognition judgments on net income. H2 predicts that example type and case interact in determining income-recognition judgments, such that the probability of making an income-increasing judgment for a revenue (expense) case is greater after receiving an affirmative (counter) example than after receiving a counter (affirmative) example. Participants judge the probability that the transaction in question qualifies for recognition of revenue or expense, so to analyze H2, the dependent variable needs to be converted to the probability that the recognition judgment is income increasing. In the revenue case the dependent variable does not require a transformation because judging a higher probability of qualifying for revenue recognition is equivalent to judging a higher probability of increasing income. However, in the expense case, judging a higher probability of qualifying for expense recognition implies a lower probability of increasing income. Therefore, we subtract the probability of qualifying for expense recognition from 100 to obtain the probability of making an income-increasing judgment. The transformed means are presented in table 2, panel A for pre-example judgments, post-example judgments, and the difference between pre- and post-example judgments, and are shown in figure 1 for post-example judgments and the difference between pre- and post-example judgments. Using the transformed post-example judgment, a GLM procedure indicates a significant case by example interaction (F = 11.37, p = 0.001). Table 2, panel B provides the complete analysis of variance (ANOVA) table. Focusing only on the revenue case, simple-effects tests indicate that income-increasing judgments are significantly more probable when participants are provided an affirmative example than when they are provided a counter example (t = 3.07, p = 0.001). Focusing only on the expense case, simple-effects tests indicate that income-increasing judgments are significantly more probable when participants are provided a counter example than when they are provided an affirmative example (t = 1.67, p = 0.049). 10 These results support H2. The main effect for case is insignificant (F = 0.03, p = 0.858), as is the main effect for example (F = 1.13, p = 0.289) Alternatively, focusing only on affirmative examples, simple-effects tests indicate that income-increasing judgments are significantly more probable when participants are provided a revenue case than an expense case (t = 2.31, p = 0.011). Focusing only on counter examples, simple-effects tests indicate that income-increasing judgments are significantly more probable when participants are provided an expense case than a revenue case (t = 2.46, p = 0.008). 11 We make no prediction about whether the absolute differences between pre- and postexample judgments differ between case and example types. The data reported in table 2, panel A suggest a larger absolute difference for the revenue case than for the expense case and for affirmative examples than for counter examples, which implies that participants react

14 712 S. CLOR-PROELL AND M. W. NELSON Panel A: Cell means TABLE 2 Results of Experiment 1: Implied Income Effects Revenue Case Expense Case Collapsed across Case Pre- Post- Pre- Post- Pre- Postexample example example example example example Example Type N Judgment Judgment Diff. N Judgment Judgment Diff. Judgment Judgment Diff. Affirmative Counter Collapsed by type Panel B: Test of H2 Sum of Mean Source df Squares Square F Value p-value Case Example Case Example 1 8, , Error , Simple effect of example for revenue-recognition case: Standard p-value Parameter Estimate Error t Value (One-Tailed) Revenue Simple effect of example for expense-recognition case: Standard p-value Parameter Estimate Error t Value (One-Tailed) Expense Participants in experiment 1 judge the probability that the transaction in the case qualifies for revenue or expense recognition. This table transforms results reported in table 1 to report descriptive statistics and hypothesis tests about the probability of making an income-increasing judgment implied by participants judgments. For the revenue case the means indicate the probability of qualifying for revenue recognition. For the expense case the means indicate 100 the probability of qualifying for expense recognition. Participants receive a basic revenue (expense) recognition standard and apply it to a case that always contains an equal mix of facts supporting recognition and facts supporting nonrecognition. Participants make pre-example judgments (based on only the case information and general standard) and post-example judgments (after receiving either an affirmative example in which all facts favor recognition and recognition is allowed, or a counter example where all facts favor nonrecognition and recognition is not allowed), with the example serving as implementation guidance that supplements the standard.

15 EXAMPLE-BASED REASONING 713 FIG. 1. Results: Experiment 1. Participants in experiment 1 judge the probability that the transaction in the case qualifies for revenue or expense recognition. Participants receive a basic revenue (expense) recognition standard and apply it to a case that always contains an equal mix of facts supporting recognition and facts supporting nonrecognition. Participants make pre-example judgments (based on only the case information and general standard) and post-example judgments (after receiving either an affirmative example in which all facts favor recognition and recognition was allowed, or a counter example where all facts favor nonrecognition and recognition is not allowed), with the example serving as implementation guidance that supplements the standard. This figure shows mean judgments of the appropriateness of making an income-increasing judgment. For the revenue case the means indicate the probability of qualifying for revenue recognition. For the expense case the means indicate 100 the probability of qualifying for expense recognition. Difference is mean post-judgment less mean pre-judgment Testing Alternative Explanations Effect of reporting threshold implied by example. One alternative explanation for the results is that an example provided as implementation more strongly to examples in the revenue case than in the expense case, and to affirmative examples than to counter examples. To test this possibility, we perform an ANOVA in which the absolute value of the difference is the dependent variable and case and example type are the independent variables. The results reveal insignificant main effects for case (F = 0.99, p = 0.322) and example type (F = 0.72, p = 0.398), and an insignificant interaction (F = 0.18, p = 0.674). This analysis indicates that participants do not react significantly differently to examples in the revenue and expense cases and do not react significantly differently to affirmative and counter examples.

16 714 S. CLOR-PROELL AND M. W. NELSON guidance could communicate a threshold necessary for a particular accounting treatment to be allowed. As discussed previously, the examples used in experiment 1 are relatively extreme, with affirmative examples indicating the accounting treatment is appropriate when all of the four facts in the example favor the treatment, and counter examples indicating the accounting treatment is not appropriate when none of the four facts in the example favor the treatment. To the extent that these examples communicate a threshold, affirmative examples communicate a high threshold for recognition (i.e., implying if have all facts favoring recognition, can recognize ), while counter examples communicate a low threshold (i.e., implying if don t have any facts favoring recognition, do not recognize ). Therefore, threshold-based reasoning biases against support for our psychological explanation, because practitioners are likely to judge recognition as more appropriate when provided a counter example (that implies a low threshold) than an affirmative example (that implies a high threshold), such that income is lower when affirmative examples are provided in the revenue case and when counter examples are provided in the expense case. In fact, threshold-based reasoning would produce a main effect of example type on judged appropriateness of recognition that is opposite to that which was predicted by H1 and supported by our results, and would produce an interaction between example type and case on the income resulting from participants judgments that is opposite to that predicted by H2 and supported by our data. Thus, the data do not support a threshold-based explanation for the results Effect of nonaccuracy incentives. Application of an example to a reporting decision can also be affected by preferences that a practitioner has for particular effects on net income. For example, many practitioners might be accustomed to preferring income-increasing alternatives. We can test in two ways for an effect of practitioners perceptions of reporting incentives. First, recall that our instructions are designed to convey an accuracy incentive along with a case that suggests a 50% probability of qualifying for recognition. To the extent that participants knowledge of incentives or reporting practices affects their judgment within the experiment, we expect their pre-example judgment to deviate from 50%. Of the 48 participants who do not select 50% as their pre-example judgment, 27 receive the revenue case and provide a mean pre-example judgment of 34.8% likelihood of revenue recognition, and 21 receive the expense case and provide a mean pre-example judgment of 55.6% likelihood of expense recognition, indicating a slight preference for conservative reporting among those who deviate from a 50% pre-example judgment. However, as discussed in footnote 7, results of hypothesis tests do not depend on whether participants express a 50% pre-example judgment. Thus, analyses of pre-example judgments do not suggest an intrusion of knowledge or reporting incentives that affects our results. Second, our design allows us to test for an incentive-based pattern of post-example judgments. Participants who perceive an income-increasing

17 EXAMPLE-BASED REASONING 715 incentive tend to recognize revenue and not expense, and participants who perceive an income-decreasing incentive tend to recognize expense and not income. Therefore, incentive-based reasoning would produce a main effect of case and no main effect or interaction between case and example type both on judged appropriateness of recognition and on the income resulting from participants judgments. Results of tests of H1 and H2 show no main effect for case, but rather the predicted main effect for example type for H1 and the predicted interaction between example type and case for H2. Thus, the data do not support an incentive-based explanation for the results Effect of individual facts indicating recognition or nonrecognition. Two of the four facts presented in the revenue and expense cases support recognition, while the other two do not support recognition. Whether a particular fact supports recognition is manipulated between subjects in four combinations of facts. To ensure that the results are not driven by any specific combination of facts, we analyze the transformed post-example judgments in a model that includes example, case, and a four-level categorical pattern variable. The analysis reveals an insignificant three-way interaction between case, example, and pattern (F = 0.43, p = 0.855), and our predicted significant interaction between case and example (F = 9.44, p = 0.003), indicating that our primary result does not depend on the specific fact pattern provided to participants Evidence About Process Used to Make Judgment. To provide insight about process, a debriefing question elicits participants perception of the process they use to make their recognition judgment. Participants are asked to choose one of three alternatives that best describes how they use the example provided in the standard to decide whether to recognize revenue or expense. One option describes a similarity-based process: I based my judgment on the amount of similarity that existed between the CAP case and the example provided in the standard. I thought the CAP case was somewhat similar to the example, so I thought it was more likely that the CAP case should be treated similar to how the example was treated. A second option describes a priming-based process: Regardless of the degree of similarity between the CAP case and the example, I based my decision on whether revenue (expense) recognition was allowed for the example. If revenue (expense) recognition was allowed for the example, I concluded it was more likely that revenue (expense) recognition would be allowed in general. If revenue (expense) recognition was not allowed in the example, I concluded it was less likely that revenue (expense) recognition would be allowed in general. A third option was Other (please explain). The order of options 1 and 2 is balanced between subjects. One participant does not indicate which process he uses to make his judgment. Sixty-two (50%) of the remaining 124 participants indicate they use a similarity-based process, 44 participants (35%) indicate a primingbased process, and 18 participants (15%) indicate another process. Including a three-level categorical variable for judgment process in our analysis of the income implied by post-example judgments, we find a significant

18 716 S. CLOR-PROELL AND M. W. NELSON three-way interaction among case, example, and process (F = 2.28, p = 0.041). Further analyses indicate that the three-way interaction occurs because the interaction between case and example is present for participants who indicate a priming- or similarity-based process, but not present for participants who indicate another process. 12 This analysis suggests that a selfreported similarity-based or priming-based process is consistent with our results. Participants who self-reported some other process provide a different pattern of results, indicating that the process variable is at least somewhat informative as to the process underlying our results. 3.3 DISCUSSION The results of experiment 1 provide support for example-based reasoning, in that participants who receive an affirmative example are more likely to conclude that revenue or expense recognition is appropriate for the case than are participants who receive a counter example. As a consequence, income is higher (lower) when affirmative examples are provided for revenue- (expense-) recognition cases than when counter examples were provided. These results are different from those that would be predicted by thresholdbased or incentive-based reasoning. Additional analysis reveal that participants who report using either a similarity-based process or a priming-based process provide results that are consistent with our predictions. 4. Experiment 2 In experiment 1, student participants are used to enhance experimental power by avoiding intrusion from experienced practitioners knowledge of existing standards and precedents. However, it is also important to determine the extent to which results generalize to circumstances where experienced practitioners existing knowledge combines with provided examples when applying implementation guidance. Therefore, while we do not posit particular effects of experienced practitioners knowledge, one purpose of experiment 2 is to generalize the results of H1 and H2 to a more experienced population. The pattern of results in experiment 1 also suggests a potential means for encouraging accurate reporting. Specifically, practitioners could be provided with both affirmative and counter examples. As predicted by H3, so long as practitioners focus equally on both types of examples, effects of example-based reasoning are counterbalanced. We focus on only the revenue case in experiment 2 because the results of experiment 1 indicate 12 Specifically, if we exclude the 18 participants who self-reported an other process, such that process is now a two-level indicator variable discriminating between a priming- and a similarity-based process, we find an insignificant three-way interaction among case, example, and process; a significant case by example interaction; and insignificant main effects of process, case, and example. The interaction between case and example is present for both the similaritybased- and priming-based-process participants when analyzed separately.