Auditor Mindsets and Audits of Complex Estimates. Emily E. Griffith University of Georgia

Transcription

1 Auditor Mindsets and Audits of Complex Estimates Emily E. Griffith University of Georgia Jacqueline S. Hammersley University of Georgia Kathryn Kadous Emory University Donald Young Georgia Institute of Technology First Version: October 8, 2013 This Version: February 24, 2014 We thank Mike Ricci and Dan Zhou for research assistance, and we thank the auditors and their firms who have given their time and effort to participate in this study. This paper has benefited from helpful comments from Michael Bamber, Jeremy Bentley, Sarah Bonner, Bryan Church, Patrick Hurley, Lisa Koonce, Bob Libby, Eldar Maksymov, Bob Mocadlo, Mark Nelson, Mark Peecher, Bob Swieringa, and from workshop and conference participants at Cornell University, the University of Georgia, the University of Southern California, and the 2014 AAA Auditing Section Mid-Year Meeting.

2 Auditor Mindsets and Audits of Complex Estimates ABSTRACT Auditors experience significant problems auditing complex accounting estimates, and this increasingly puts financial reporting quality at risk. Based on analyses of the specific errors that auditors commit, we propose that auditors need to be able to think more broadly and incorporate information from a variety of sources in order to improve audit quality for these important accounts. We experimentally demonstrate that a deliberative mindset intervention improves auditors ability to identify unreasonable estimates by improving their ability to identify and incorporate into their analyses contradictory information from other parts of the audit. We perform additional analyses to demonstrate that our intervention improves auditor performance by causing them to think differently rather than simply to work harder. We demonstrate that thinking more broadly can improve the identification of unreasonable estimates and, in doing so, provide new directions for addressing audit quality issues. Key Words: Accounting estimates, audit quality, fair value, impairment, mindset, professional skepticism

3 Auditor Mindsets and Audits of Complex Estimates 1. Introduction Complex accounting estimates, including fair values, impairments, and valuation allowances, are increasingly important to financial statements (e.g., Barth [2006]). However, auditors experience significant difficulty in auditing complex estimates, suggesting that audit quality may be low in this area (Public Company Accounting Oversight Board (PCAOB) [2010e], Griffith, Hammersley, and Kadous [2014], Bratten, Gaynor, McDaniel, Montague, and Sierra [2013], Cannon and Bedard [2014]). Some of the difficulties auditors experience can be attributed to high levels of uncertainty about valuations given volatile financial markets and innovative securities (Christensen, Glover, and Wood [2012], Bratten et al. [2013], Cannon and Bedard [2014]). However, others arise from problems with auditor judgment and the audit process. Analysis of PCAOB inspection reports for the largest accounting firms reveals that fair value measurements, including impairments, and other estimates are among the most frequently cited accounts for auditor errors, and that, while other audit deficiencies have decreased over time, deficiencies involving fair values and impairments have not (Church and Shefchik [2012]). Chief among auditors judgment problems associated with auditing complex estimates are that auditors fail to adequately test the data and assumptions underlying management s estimates and they fail to notice and incorporate into their analysis inconsistencies among the assumptions, other internal data, and external conditions (Griffith et al. [2014]). These types of shortcomings result in over-reliance on management s process for generating estimates (Griffith et al. [2014]), which implies that any management bias in an estimate may survive the audit. Consistent with this concern, financial statement users tend to view estimates as less reliable than 1

4 other financial statement items, and they discount them accordingly (Richardson, Sloan, Soliman, and Tuna [2005], Lev, Li, and Sougiannis [2010]). In this paper, we examine whether and how changing auditors mindsets can improve audits of estimates, thereby enhancing audit quality in this important area. Mindsets are the judgment criteria, cognitive processes, and procedures that produce a disposition or readiness to respond in a certain manner (Hamilton, Vohs, Sellier, and Meyvis [2011], Gollwitzer [1990]). Mindsets have two essential features: (1) the cognitive processes and procedures that make them up are more general than what is required for the task at hand and (2) once activated, mindsets remain active beyond the initial task to influence subsequent tasks (Gollwitzer [1990], Hamilton et al. [2011]). Thus, mindsets are not merely a template for approaching a particular type of task; they represent a more global readiness to respond in a particular way. Indeed, mindsets are independent of the particular goal to which they are directed and of the semantic content to which the mindset s criteria, processes, and procedures are applied (Wyer and Xu [2010]). 1 Auditors over-reliance on management s process for generating estimates arises in part from structural features of the audit context, including audit firms choice to audit management s model and their use of step-by-step audit programs (Griffith et al. [2014]). We propose that these features cause auditors to approach the audit of estimates in a mindset that encourages efficient completion of a verification task rather than one that encourages critical thinking and 1 Prior auditing research uses the term mindset in varying ways. One stream of research examines prompts designed to increase auditors awareness of specific risks such as heightened fraud risk or low client integrity and labels these prompts as mindsets (e.g., fraud mindset, skepticism mindset) (Hammersley, Bamber, and Carpenter [2010], Bowlin, Hobson, and Piercey [2013]). These prompts are not independent of the tasks under study. A second stream of research examines judgment frameworks based on psychological theories related to mindsets (Rasso [2013], Backof, Bamber, and Carpenter [2013a], Backof, Thayer, and Carpenter [2013b]). These judgment frameworks provide task-specific instructions and documentation requirements. Task independence is a critical feature of the definition of mindset we employ in this study. We design our study to ensure the mindset intervention is independent of the experimental task to isolate the effect of mindset on auditor judgments from factors that are specific to the experimental task. 2

5 consideration of a broad set of evidence. We examine whether prompting auditors to adopt a deliberative mindset can improve auditor judgments in this area. We focus on auditors failure to identify and incorporate contradictory information into their analyses because this is a key judgment problem undermining audit quality in this area (PCAOB [2010e], Griffith et al. [2014]). Moreover, it is not a problem that can be readily addressed by writing new standards or creating more comprehensive checklists. That is, all information that is relevant to a key assumption underlying an estimate is not necessarily known or knowable while planning an evaluation of the assumption. Some such information will be available only incidentally in another part of the audit or in external information collected for another purpose. Auditors must be able to recognize the relevance of the incidentally presented information and must be open to incorporating it into analysis of the assumption. We develop theory predicting that putting auditors in a deliberative mindset can improve audits of complex estimates in realistic settings in which all the relevant information is not provided to auditors directly. In particular, a deliberative mindset facilitates broad focus of attention (Gollwitzer and Bayer [1999]), recognition of incidental information (Fujita, Gollwitzer, and Oettingen [2007]), and impartial information processing (Beckman and Gollwitzer [1987]). A deliberative mindset thus should improve auditors ability to identify an unreasonable estimate by facilitating auditors consideration of a broad set of evidence, facilitating their identification of inconsistencies between assumptions and data by allowing them to hold open the decision space while moving from one assumption to the next, and prompting a critical analysis of issues underlying management s estimate. We conduct an experiment in which we assign 94 senior-level auditors from three Big Four audit firms to one of three mindset conditions (deliberative, implemental, or control) and 3

6 ask them to audit a client s step one analysis of a goodwill impairment test. 2 In our case, the client concluded that the calculated fair value of its business unit exceeds book value and thus has no goodwill impairment. However, the case contains seeded errors and inconsistencies among certain assumptions that imply the step one analysis is biased and overstates the fair value. Consequently, goodwill may be impaired. We expect that auditors in a deliberative mindset will be more critical of the client s fair value analysis than auditors in other conditions because they will be more likely to identify the seeded issues. We find evidence consistent with our predictions. Auditors in a deliberative mindset assess the client s biased fair value as less reasonable than do auditors in control and implemental mindset conditions. Auditors in a deliberative mindset are also more likely to choose a next action step that reflects more urgent concern that the fair value is unreasonable. Finally, auditors explanations for their decisions are more likely to include the seeded issues and more valid issues with the estimate, generally, when they are in a deliberative mindset than when they are not. Additional analyses demonstrate that auditors in a deliberative mindset are not less trusting of management in general, but they target the specific assumptions with seeded errors. They also appear to evaluate evidence about the appropriateness of the aggressive discount rate more critically than do auditors in other conditions. Mediation analyses and structural equation modeling confirm the posited mechanism is at work. That is, deliberative mindset auditors identification of the seeded inconsistencies and their evaluations of the appropriateness of the discount rate jointly fully mediate the relationship between the deliberative mindset and the 2 The key to understanding auditor performance, and thus audit quality, in a complex setting is identification of the cognitive mechanisms, and experiments have a comparative advantage in abstracting critical elements of real world settings and controlling for non-critical elements, allowing them to isolate relevant mechanisms (Libby and Luft [1993], Bonner [1999]). 4

7 assessed reasonableness of the fair value. Thus, the deliberative mindset intervention facilitates the identification of the seeded issues and critical analysis of the discount rate, which increases concern about the reasonableness of the fair value. Our study provides new direction for improving audits of complex estimates, and it adds to growing evidence that improved critical thinking, rather than increased doubt or increased demand for evidence, is key for improving audit quality. Our view that improved critical thinking is key to improved audit quality is consistent with demonstrations in the area of fraud detection that increased doubt can increase the amount of planned audit work without increasing audit quality (Hammersley [2011], Hammersley, Johnstone, and Kadous [2011]), whereas auditor identification of the specific problems underlying the numbers and targeted collection of additional evidence does increase audit quality (Hammersley et al. [2011], Simon [2012]). Further, audit quality in these areas is improved by interventions designed to directly address auditors critical thinking (Hoffman and Zimbelman [2009], Brewster [2011], Simon [2012]). The decision problems that auditors experience with complex estimates, including overreliance on management s process, failure to notice discrepancies and inconsistencies, and failure to gather sufficient evidence, are generally viewed as resulting from insufficient professional skepticism (e.g., PCAOB [2011], see also Hurtt, Brown-Liburd, Earley, and Krishnamoorthy [2013]). 3 While auditing standards define professional skepticism at a conceptual level as an attitude that includes a questioning mind and a critical assessment of evidence (AU , American Institute of Certified Public Accountants (AICPA) [1997]), this often translates operationally into the amount of evidence demanded (e.g., see Glover and 3 Another view is that an increasingly difficult task, caused by environmental and task factors, is largely to blame for auditors poor performance (e.g., Christensen et al. [2012]; Bell and Griffin [2012]; Bratten et al. [2013]). This view is consistent with our own, insofar as it suggests that a solution lies in changing the task (or at least the approach to the task), rather than changing decision maker characteristics. 5

8 Prawitt [2013]). Indeed, the academic literature defines professional skepticism as a propensity to defer reaching a conclusion until sufficient evidence is accumulated or as a propensity to view the risk of misstatement as heightened (Nelson [2009], Quadackers, Groot, and Wright [2013]). While these views vary in their starting positions (i.e., neutral belief versus presumptive doubt), they both view a more skeptical auditor as needing more evidence to be convinced that management s estimates are accurate (e.g., Hurtt [2010]). We demonstrate that a focus on the critical assessment of evidence is crucial for improving audits of estimates. That is, our view is that in auditing complex estimates, auditors tend to collect enough evidence, but they do not approach the evidence in a manner conducive to critical thinking. We show that changing auditors mindsets through a brief intervention allows them to make better use of the available information. 2. Background and Hypothesis Development 2.1 BACKGROUND The majority of research attempting to improve auditor skepticism has focused on auditors skeptical judgment (an understanding that additional work or effort is needed), and does not test whether skeptical judgment translates into skeptical action (a change in behavior based on the skeptical judgment) (Hurtt et al. [2013]). We view critical thinking as a key determinant of whether auditors know what skeptical action to take, and so whether they can take appropriate skeptical action. Concurrent research has started to examine methods aimed at increasing auditors critical thinking. For example, Plumlee, Rixom, and Rosman [2012] train auditors to be more skeptical by thinking critically. This method has not yet been applied to audits of estimates, but could be productive there. In the area of auditing estimates, Maksymov, Nelson, and Kinney [2013] show that the framing of audit procedures affects how much time 6

9 auditors allocate to less verifiable procedures, such as those involved in auditing assumptions underlying estimates. While Maksymov et al. [2013] stop at skeptical judgment, we expect this type of framing to be beneficial in encouraging skeptical actions, as well, as it changes how auditors think about procedures. Closer to our own research, Backof et al. [2013b] and Rasso [2013] impose judgment frameworks (task-specific instruction) to change auditors responses to evidence. 4 This is an important step judgment frameworks can effectively instruct auditors to perform a task in a specified way and to document various considerations. However, it is unclear whether these frameworks affect auditors critical thinking about the available evidence or simply provide them with specific cues to a desired answer. Thus, it is unclear whether auditor judgments are really improved (or are merely more conservative) and, if improved, whether the improvement generalizes to other situations and whether it can be sustained. Our mindset intervention conveys no task-specific instruction, requires no extra work of auditors, and so is general. We designed our experimental task to allow insight into the critical thinking mechanisms at work. We focus our examination on a critical judgment problem that has been a frequent topic of PCAOB inspection comments on complex estimates auditors failure to notice and incorporate into their analysis inconsistencies among the assumptions, other internal data, and external conditions. For example, the PCAOB noted in one inspection report that when determining the fair value of a business unit for two different purposes, management and the audit firm s specialist generated two very different values, and there was no evidence that the auditors considered the difference between the two measures (PCAOB [2010c], p. 10). Other reports note auditors failing to reconcile values used in management s models with industry data 4 Rasso [2013] uses mindset terminology to describe his manipulation; however, the manipulation does not possess the two essential characteristics of a mindset (e.g., Gollwitzer [1990]). His manipulation best fits the description of a judgment framework, so we refer to it as such. 7

10 (PCAOB [2010b], p. 4), failing to consider customer growth and retention data and the feasibility of forecasted changes in cost levels in evaluating revenue projections for a goodwill impairment analysis (PCAOB [2010a], p. 10), and failing to reconcile a company s capital needs projections with the contradictory capital projections used in a goodwill impairment analysis (PCAOB [2010d], p. 6-7). 2.2 MINDSET THEORY AND HYPOTHESIS DEVELOPMENT We argue that a change in auditor mindset can improve auditors identification of such inconsistencies among pieces of evidence and thus audit quality in the area of complex estimates. A mindset is a set of judgmental criteria and cognitive processes and procedures that produce a disposition or readiness to respond in a certain manner (Gollwitzer [1990], Hamilton et al. [2011]). For example, auditors might find themselves in a mindset that encourages a broad consideration of information in order to make the best choice from a set of alternatives, or they might find themselves in a mindset that encourages a linear processing of obviously task-relevant information in order to complete a task quickly. A critical feature of mindsets is that they are not specific to the task at hand (Gollwitzer [1990]), but instead represent a global readiness to respond (Hamilton et al. [2011]). Mindsets influence how individuals approach their tasks, including what aspects of the task they attend to and what their ultimate judgments will be. Thus, the influence of mindsets on judgments is profound. Moreover, once activated, a mindset persists for some time and across unrelated tasks without conscious awareness (Gollwitzer [1990], Wyer and Xu [2010]). This implies that mindsets that have been instantiated in service of a particular task in a particular setting can affect decision behavior in subsequent tasks, even though the goal and content of the tasks are very different (Wyer and Xu [2010]). For example, individuals who deliberate the pros and cons 8

11 of alternative possible decisions related to an unresolved personal problem apply a similar deliberative process when performing a subsequent unrelated task. When asked to write the ending to a fairy tale, they describe characters that are relatively ambivalent and unable to decide what to do (Gollwitzer, Heckhausen, and Steller [1990]). In contrast, individuals who describe how to implement a course of action related to an unresolved personal problem as their first task subsequently write fairy tales about characters that are more implemental in that they decide on and pursue courses of action (Gollwitzer et al. [1990]). There are numerous ways of categorizing mindsets (Wyer and Xu [2010]), but we focus on the deliberative and implemental mindsets as described by Action Phase Theory (Gollwitzer [1990]; Gollwitzer et al. [1990]). Action Phase Theory asserts that deliberative and implemental mindsets are differentially helpful to different decision phases, and may arise naturally in response to certain tasks. Decision makers tend to adopt a deliberative mindset when deciding what action to take or whether an action should be taken. A deliberative mindset facilitates a broad consideration of the pros and cons of various alternatives before action. Once an action has been determined, decision makers tend to adopt an implemental mindset, which facilitates efficient processing of tasks without much consideration of higher goals. An implemental mindset facilitates planning how, rather than whether, to execute a task or reach a goal, which results in increased efficiency. Mindsets affect the acquisition and processing of information. In general, mindsets promote the acquisition of mindset-consistent information (Gollwitzer et al. [1990]). Specifically, a deliberative mindset fosters a wide focus of attention that expands beyond information that is strictly relevant to a task (Gollwitzer and Bayer [1999], Heckhausen and Gollwitzer [1987]). It also increases recognition of incidentally presented information (Fujita et 9

12 al. [2007]) and promotes impartial information processing (Beckman and Gollwitzer [1987], Taylor and Gollwitzer [1995]). In contrast, an implemental mindset fosters a narrow focus on obviously task-specific information and biased information processing that only considers information that supports the chosen goal or preference (Gollwitzer and Bayer [1999], Heckhausen and Gollwitzer [1987], Henderson, De Liver, and Gollwitzer [2008]). When auditing complex estimates, auditors appear to adopt a mindset well suited to the goal of verifying accounts and ill suited to critical analysis and broad thinking. That is, auditors appear to approach audits of estimates as verification tasks, with a goal of finding evidence to support each element (assumption) of the estimate separately, then moving on to the next element (Griffith et al. [2014]). We expect that this task framing causes auditors to adopt a mindset more like the implemental, rather than deliberative, mindset. This makes it difficult for them to notice and process external and internal information that is presented incidentally and is inconsistent with management s estimate. Consistent with this conclusion, PCAOB inspections reveal that significant problems occur in audits of fair values in that auditors failed to evaluate the effect of contradictory evidence when concluding on the reasonableness of certain significant assumptions in such audits (PCAOB [2010e], p. 12). The above suggests that auditors tend to default to a mindset that is more similar to the implemental than to the deliberative mindset when auditing estimates. An implemental mindset orients a person to quick, decisive action in order to complete a task. Thus, it facilitates efficient completion of pre-programmed tasks, such as verification that evidence exists for a given management assertion. When auditing estimates, an implemental mindset likely prompts auditors to focus on the evidence obtained for the purpose of testing a particular assumption, which can help auditors efficiently complete each step of evaluating an assumption individually, 10

13 then move on to evaluate the next assumption. However, we expect an implemental mindset is less well suited to tasks that require critical thinking and a broad consideration of information before taking action, such as incorporating relevant but incidentally presented information when evaluating the assumptions underlying estimates collectively (i.e., holding open the decision space) and evaluating assumptions critically (i.e., addressing is this a good assumption? rather than is this assumption acceptable? ). Mindsets can be changed with external interventions (Gollwitzer et al. [1990]), and we propose that changing auditors mindset to one that encourages deliberative, critical thinking could improve the quality of audits of estimates. In general, an implemental mindset causes attention to be centrally focused on the specific task at hand, and, particularly, to one-sided processing of that information, while a deliberative mindset increases open-mindedness to incidentally presented information and a more balanced view of that evidence. Since auditors approach audits of complex estimates as a series of verification tasks such as evaluating individual assumptions, a confirmatory focus on task-central information likely limits both auditors ability to consider relevant evidence when it appears outside the task (audit program step) of evaluating a particular assumption. In particular, this focus likely limits their ability to recognize contradictions and inconsistencies among different assumptions and between the assumption and the underlying evidence. We believe that it is common that key information for evaluating an assumption is not part of the management-provided information set for that assumption, and PCAOB inspection deficiencies are consistent with this belief (e.g., PCAOB [2010a], p. 10; PCAOB [2010b], p. 4). We hypothesize that a deliberative mindset will help auditors acquire and incorporate such incidentally presented information into their judgments about complex estimates. This implies that auditors in a deliberative mindset will be more likely to recognize seeded errors and 11

14 inconsistencies underlying complex estimates, and they will take these into account when assessing the reasonableness of the estimate, assessing a biased estimate as less reasonable. Because auditors in a deliberative mindset have specific ideas about the issues and problems underlying the estimate, they should recognize the urgency of the situation and be more likely to bring the estimate to their supervisor s immediate attention. We present these hypotheses below. Hypothesis 1: Auditors receiving a deliberative mindset prompt will evaluate a biased estimate as less reasonable than will auditors not receiving the prompt. Hypothesis 2: Auditors receiving a deliberative mindset prompt are more likely to call a biased estimate to their manager s attention immediately than are auditors not receiving the prompt. Hypothesis 3: Auditors receiving a deliberative mindset prompt will identify more seeded inconsistencies and errors underlying a biased estimate than will auditors not receiving the prompt. Although we expect a deliberative mindset prompt to facilitate identification and incorporation of incidentally presented information into one s analyses, as well as unbiased information processing for complex estimates, we note that an implemental mindset prompt may also prove useful for some aspects of auditing complex estimates relative to the control condition. To the extent that auditors in the study are not already in an implemental mindset, an implemental mindset prompt may facilitate efficient task completion and provide a framework for taking action. To the extent that action steps are clear, an implemental mindset prompt will motivate auditors to efficiently engage in those steps (e.g., Freitas, Gollwitzer, and Trope [2004]) and will facilitate evaluation of information in a single direction (Henderson et al. [2008]). Further, by suppressing deliberation about whether the task is worth engaging in, it may increase persistence (Brandstätter and Frank [2002]) and performance (Armor and Taylor [2003]). 12

15 3. Method We test our hypotheses in an experiment in which we manipulate mindset between participants at three levels (deliberative, implemental, and control/no mindset manipulation). We obtained 94 usable responses from experienced audit seniors who participated while attending national or local training sessions sponsored by their firms. 5 Participants come from three Big 4 firms and their audit experience ranges from 30 to 72 months (average 39.3 months). Seventyeight percent are CPAs. Evaluating assumptions related to complex estimates is most often performed by audit seniors (Griffith et al. [2014]), so these auditors are appropriate participants. 3.1 TASK Participating auditors evaluated the reasonableness of the fair value of a business unit of an electronics manufacturer. The fair value is used in the client s step one analysis of the goodwill impairment test on the current year audit. The client used a discounted cash flow model to estimate the fair value. Projections of future revenue, operating expenses, and capital expenditures, and the discount rate are important inputs into the discounted cash flow model. Auditors were told that their team had already evaluated the appropriateness and mathematical accuracy of the model, and that their job was to evaluate these four assumptions and form a preliminary conclusion about the reasonableness of the fair value. The case, which was adapted from a firm s training materials, included background information about the company, followed by the goodwill impairment test that had been prepared by the client, including the step one analysis and the discounted cash flow model supporting the analysis. The case also included four sections with summaries of the audit evidence related to each of the four key assumptions: revenue projections, expense projections, capital expenditures 5 We received 103 complete instruments from auditors meeting our predetermined criteria of having at least 30 months of audit experience. We eliminated data from 9 participants who indicated that they did not give the materials sufficient attention. 13

16 projections, and the discount rate. We seeded four issues indicating problems with the fair value throughout the case. These issues and their potential impacts on the fair value are summarized in Table 1. Importantly, a key piece of information necessary to identify each seeded issue appears as part of the evidence related to an assumption other than the assumption implicated by the issue. This setting requires auditors to recognize the incidentally presented information as relevant to the other, targeted assumption to identify the seeded problems. It also requires them to incorporate the information into their analysis, even though it contradicts information presented that is labeled as relevant for the targeted assumption. For example, identifying the seeded issue related to capital expenditures requires noticing and incorporating a piece of information presented among the evidence supporting management s operating expense projections that contradicts the capital expenditures assumption. [Insert Table 1 here] After reading the case, auditors assessed the reasonableness of the fair value, decided what they would like to do next regarding the fair value, and gave reasons for their decision. Next, they put away the case and opened a packet containing additional questions, including evaluations of the individual assumptions and demographic information. 3.2 INDEPENDENT VARIABLE We manipulated mindset at three levels: deliberative, implemental, and control. In the deliberative and implemental mindset conditions, a mindset intervention preceded the case materials described above. Auditors assigned to the deliberative mindset condition were instructed to consider the possibility of doing an international rotation with their firm and to list three pros and three cons of doing such a rotation, thus encouraging them to deliberate about the merits of such a rotation. Auditors assigned to the implemental mindset condition were 14

17 instructed to assume that they had decided to pursue a one-year international rotation and were asked to list six steps that they would take to obtain a rotation, thus encouraging them to think about how they would implement a strategy to obtain such a rotation. Auditors assigned to the control condition started immediately on the case materials described above. They completed the implemental mindset intervention after completing the case materials in order to equalize timing across all three conditions. Our mindset intervention instructions are briefer than those used in the prior psychology literature (e.g. Gollwitzer and Kinney [1989], Taylor and Gollwitzer [1995], Gollwitzer and Bayer [1999]); however, they retain the essential features of those instructions. In particular, participants in the deliberative mindset condition were asked to list the pros and cons of achieving a potential goal (specifically, three pros and three cons of going on an international rotation), whereas participants in the implemental mindset condition were asked to list the steps they would take to accomplish a goal (specifically, six steps to obtain an international rotation). While prior research has participants choose their own dilemma (in the deliberative condition) or goal (in the implemental condition), we used the international rotation to establish these for them. This procedure considerably reduces the duration of the task, as participants don t have to contemplate which goal to consider, while ensuring the goal is relevant to them as auditors yet unrelated to the primary audit task. 6 6 We validated our manipulation in a pilot study. Prior research finds that deliberative and implemental mindsets affect attitude strength in that participants in a deliberative mindset are more ambivalent about issues and less likely to express a one-sided attitude towards a particular issue (Henderson et al. [2008]). Therefore, to validate our international rotation version of the mindset manipulation, we gave senior-level auditing students the international rotation manipulation (modified to international internship ) and measured their attitude ambivalence using the same 29 questions used in Henderson et al. [2008]. Results indicated the abbreviated manipulation was effective: the deliberative mindset group evidenced a significantly higher amount of ambivalence (mean (SD) = 3.34 (0.62) than the implemental mindset (mean (SD) = 2.87 (0.67), t 39 = 2.36, p = 0.024) and control (mean (SD) = 2.96 (0.73), t 41 = 1.84, p = 0.073, two-tailed) groups. 15

18 Importantly, our mindset intervention differs from the various auditor judgment frameworks that also draw upon mindset theories in concurrent work (e.g., Rasso [2013], Backof et al. [2013a, b]). These judgment frameworks include task-specific instructions to auditors to document certain considerations, and they provide an explicit structure for auditors to follow as they perform and document the assigned task. Thus, they do not have either of the essential features of mindsets (i.e., being more general than required by the task and carrying over from a prior task). In addition, these judgment frameworks confound distinct types of mindsets (e.g., the deliberative/implemental mindsets are conflated with construal level and counterfactual reasoning mindsets). In contrast, our mindset intervention references a deliberative or implemental mindset only, is unrelated to the task content, and it does not change the task instructions, structure, or documentation requirements as judgment frameworks necessarily do. We use this intervention to isolate the effects of deliberative and implemental mindsets on auditors cognition and processing of information in this setting and their subsequent judgments about estimates. To confirm that participants internalized the assigned mindset (deliberative or implemental), a research assistant who was blind to experimental condition and hypotheses categorized the items written about international rotations as either identifying pros and cons or listing specific steps. Of the items participants in the deliberative (implemental) condition wrote, 97.2% (98.6%) were coded as identifying pros and cons (listing specific steps). This indicates that participants attended to our mindset manipulation. 3.3 DEPENDENT VARIABLES We measured three primary dependent variables: reasonableness of the fair value, decision about what action auditors would take next, and the specific reasons or issues 16

19 supporting this decision. Auditors assessed the reasonableness of the fair value on an 11-point Likert scale anchored by 0 (not at all likely to be reasonable) to 10 (extremely likely to be reasonable). Because we seeded issues that bias the estimate upward, lower values represent better judgments. Table 1 shows that, based on the case information, any one of the four seeded issues has a potentially material impact on the fair value, and any two or more of the issues in combination have a highly material impact. We use this dependent measure to test Hypothesis 1. Auditors next chose from four options about what to do next: (1) conclude that the fair value is reasonable; (2) continue audit work under the assumption that the fair value is reasonable but delay forming a final conclusion until talking to the manager about one or more issues the next time s/he is on site; (3) do not make a conclusion but call the manager immediately regarding issues that may indicate the fair value is not reasonable; or (4) conclude that the fair value is materially overstated. No participants chose the fourth option, and only four chose the first option. 7 For analysis purposes, we recoded this variable as contact the manager immediately (option 3) or not (options 1 and 2). We use this dependent measure to test Hypothesis 2. Finally, auditors provided specific reasons supporting their decision. Two research assistants with auditing experience who were blind to hypotheses and experimental condition independently coded the reasons and issues into one of four categories: target issues, other connected issues, other issues, and items supporting the fair value. Target issues include the four issues seeded in the case that indicate problems with the fair value. These issues can only be identified by combining information about one assumption with information presented incidentally as part of the information ostensibly about another assumption. Other connected 7 Of these four, three participants were in the control condition and one participant was in the implemental condition. 17

20 issues are items other than the four seeded issues that also indicate problems with the fair value and that can only be identified by connecting different pieces of information presented within the section of the case related to the implicated assumption. For example, items questioning the appropriateness of current revenue growth assumptions based on optimistic past revenue growth assumptions are coded here. Other issues also indicate problems with the fair value, but they directly restate information from the case or evaluate information in isolation, without combination with other case information. Finally, items supporting the fair value indicate no problems with the fair value. The coders initial agreement rate was 93.8 percent and Cohen s kappa, a measure of agreement above chance, was 0.90 (p < 0.01). The coders met to resolve their coding differences and the resolved coding is used in the analysis reported here. We use the number of target issues identified as our dependent measure in tests of Hypothesis Results 4.1 PRELIMINARY ANALYSIS We estimated general linear models for each of our dependent measures with experimental condition, firm, and the interaction of condition and firm as factors. None of the main or interactive effects involving firm was significant. Therefore, we combine observations across the three firms in the following analyses. 4.2 TESTS OF HYPOTHESES Our primary tests of hypotheses contrast the deliberative mindset condition with the control condition and the implemental mindset condition, jointly. We also report comparisons of the deliberative mindset condition with each of these other conditions, singly, as well as comparisons of the implemental mindset and control conditions to each other, for further information. We expect that control condition auditors tend to be in a mindset comparable to the 18

21 implemental mindset condition, so we don t anticipate large differences between the control and implemental mindset condition; however, this design allows us to establish whether the implemental mindset intervention also provides some judgment benefits over the control condition. Hypothesis 1 examines whether auditors in a deliberative mindset are better able to recognize an unreasonable estimate than other auditors. Table 2 provides descriptive statistics (Panel A), an ANOVA model (Panel B), and contrasts, including the test of Hypothesis 1 (Panel C) for auditors assessments of how likely it is that the fair value estimate is reasonable. Panel A shows that mean reasonableness judgments appear to be lowest in the deliberative mindset condition, as expected. Panel B shows that experimental condition significantly influences assessed reasonableness (p = ). 8 [Insert Table 2 here] Planned contrasts in Panel C confirm that, as predicted, assessed reasonableness of the fair value is lower in the deliberative condition than in the other two conditions (t 91 = 2.63, p = ). Examining the comparisons separately, both the difference in reasonableness assessments between the deliberative mindset and the control condition and between the deliberative and implemental mindset conditions are significant (t 91 = 2.57, p = and t 91 = 1.79, p = , respectively). Also as expected, mean reasonableness assessments do not differ in the implemental mindset and control conditions (t 91 = 1.06, p = ). These results support Hypothesis 1 and show that a deliberative mindset helps auditors to identify biased estimates as unreasonable. 8 We report two-sided p-values for all tests other than tests of our directional predictions (comparisons of the deliberative mindset condition to other conditions, collectively and singly), for which we report one-sided p-values. 19

22 Hypothesis 2 predicts that auditors receiving the deliberative mindset intervention will be more likely to call attention to the problems they found in the fair value by contacting their manager immediately than will auditors in other conditions. This is important because a problematic fair value is more likely to be acted on in the audit if more senior team members are aware of the issue earlier. Table 3 shows descriptive statistics by condition (Panel A) and contrasts from a logistic regression model (Panel B) for the proportion of auditors choosing to contact their manager immediately. Untabulated analyses show that experimental condition significantly influences decision (Χ 2 2 = 10.52, p = ). The descriptive values show the highest value for the deliberative mindset and the lowest for the control condition, with the value for the implemental mindset condition falling between the two. [Insert Table 3 here] Planned contrasts confirm that auditors in the deliberative mindset condition are more likely than auditors in the implemental mindset and control conditions to contact their manager immediately (74% versus 46%, Z = 2.96, p = ), supporting Hypothesis 2. The contrast between the deliberative mindset and control conditions is highly significant (Z = 3.23, p = ). The proportion of auditors reporting they would contact their manager immediately in the implemental mindset condition lies between the proportions for the control and deliberative condition. That proportion is marginally significantly lower than the proportion in the deliberative condition (Z = 1.64, p = ), and it is marginally significantly higher than that in the control condition (Z = 1.86, p = ). Hypothesis 2 is supported in that the deliberative mindset intervention is associated with a higher likelihood of contacting the manager immediately, though the implemental mindset intervention also appears to provide some benefit in this regard over the control condition. 20

23 Table 4 presents descriptive statistics and contrast tests for the categories of reasons auditors gave for their decisions. Target issues, the main dependent variable for testing Hypothesis 3, is in bold, as is total valid issues. Hypothesis 3 predicts that auditors in the deliberative mindset condition will identify more target issues than will those in the implemental mindset and control conditions. The means shown in Panel A generally support this pattern. These means also show that auditors in the deliberative condition identify more valid issues with the fair value, overall, than auditors in other conditions, and they identify fewer reasons in support of the fair value than auditors in other conditions. [Insert Table 4 here] Panel B provides the planned contrasts for the number of target issues identified. Recall that there were four seeded target issues. Across all conditions, participants listed 0 to 3 of these items, with the mean (standard deviation) being 0.65 (0.73) and the median 1. Given the nature of the distribution, we used a Poisson regression to generate our contrasts. The model fit is good (Χ 2 91 = 86.04, p = ), and experimental condition is a weak predictor of the dependent measure (Χ 2 2 = 4.02, p = ). The contrasts reveal that auditors in the deliberative condition identified and incorporated into their analyses more critical issues than did auditors in other conditions (Z = 1.90, p = ). This result is driven by the contrast between the deliberative and control conditions (Z = 1.97, p = ), while the contrast between the deliberative and implemental conditions is not significant (Z = 1.07, p = ). As expected, the contrast between the implemental mindset and control conditions also is not significant (Z = 1.09, p = ). These results provide support for Hypothesis 3 in that the deliberative intervention facilitated auditors 21

24 acquisition and incorporation of key incidental information that indicated the fair value was unreasonable into their analyses. As a supplementary test of Hypothesis 3, we also examined whether the total number of valid issues identified followed the same pattern as target issues. Relevant contrasts are in Table 4, Panel C. The total number of valid issues identified varies from 0 to 8, with a mean (standard deviation) of 2.45 (1.49) and a median of 4. When considering the total number of valid issues identified, auditors in the deliberative mindset condition identified significantly more problems with the fair value than did auditors in the implemental mindset and control conditions (t 91 = 2.28, p = ). The contrast between the deliberative mindset and the control conditions is significant (t 91 = 2.20, p = ), and the contrast between the deliberative and implemental mindset conditions is marginally significant (t 91 = 1.57, p = ). Also as expected, auditors in the control and implemental mindset conditions did not identify different numbers of problems with the fair value (t 91 = 0.87, p = ). These results largely support Hypothesis 3 and are consistent with the overall results for target issues. 4.3 ADDITIONAL ANALYSES TESTS OF MECHANISM Theory suggests that the deliberative mindset intervention can help auditors to make better judgments and decisions about an unreasonable estimate by facilitating recognition and incorporation into their analyses of incidentally presented information about problems with the fair value and by prompting unbiased (critical) evaluation of information. We provide additional analyses to demonstrate that these mechanisms are at work and that the deliberative mindset intervention makes auditors think differently, rather than simply making auditors work harder or behave more conservatively. 22

25 Evidence that auditors aren t working harder, but thinking differently. First, we rule out the possibility that our deliberative mindset intervention causes auditors to work harder than they otherwise would. This is important because auditors, like other decision makers, have limited cognitive resources. This implies that having auditors work harder on one task is likely to impair their performance on the following task (e.g., see Hurley [2013]), and thus, solutions that do not increase cognitive load should be preferred over those that do. We note that our deliberative and implemental mindset manipulations are based on the psychology literature, and thus, they are independent of the experimental task. We hold task instruction constant across conditions. In addition, there is no language in our manipulations that would be expected to influence how hard auditors work. Nonetheless, we asked participants to rate how hard they worked on the task on a scale from 0 (not at all hard) to 10 (extremely hard). The overall mean (standard deviation) was 6.18 (1.40), and it did not vary across experimental conditions (F 2,91 = 0.07, p = ). Auditors in the deliberative mindset condition reported the same effort level as auditors in the other conditions (t 91 = 0.32, p = ). In addition, we timed task performance for auditors from two of the three firms. We find no difference in effort duration across experimental conditions (F 2,66 = 0.71, p = ). Auditors in the deliberative mindset condition took equally long to complete the task as did auditors in the other conditions (t 66 = 0.81, p = ) Evidence that auditors target the underlying assumptions with errors. Second, we demonstrate that deliberative mindset condition auditors assessments of the reasonableness of the specific underlying assumptions associated with management s model were targeted to the assumptions with seeded errors, ruling out the possibility that the deliberative mindset prompt simply made auditors behave more conservatively across the board. As part of their assignment 23