Clarifying conditions and decision points for mediational type inferences in Organizational Behavior y

Transcription

1 Journal of Organizational Behavior J. Organiz. Behav. 27, (2006) Published online 7 September 2006 in Wiley InterScience ( DOI: /job.406 Clarifying conditions and decision points for mediational type inferences in Organizational Behavior y JOHN E. MATHIEU* AND SCOTT R. TAYLOR University of Connecticut, Storrs, Connecticut, U.S.A. Summary Although mediational designs and analyses are quite popular in Organizational Behavior research, there is much confusion surrounding the basis of causal inferences. We review theoretical, research design, and construct validity issues that are important for drawing inferences from mediational analyses. We then distinguish between indirect effects, and partial and full mediational hypotheses and outline decision points for drawing inferences of each type. An empirical illustration is provided using structural equation modeling (SEM) techniques, and we discuss extensions and directions for future research. Copyright # 2006 John Wiley & Sons, Ltd. Introduction Over 20 years ago, Baron and Kenny (1986) and James and Brett (1984) published papers that have had a profound influence on Organizational Behavior research and theory. Those authors advanced a theoretical foundation and analytic guidelines for drawing mediational inferences theories, methods, and analyses that elucidate the underlying mechanisms linking antecedents and their consequences. At issue in this approach are research questions that seek to better understand how some antecedent (X) variable influences some criterion (Y) variable, as transmitted through some mediating (M) variable. In this sense, mediators are explanatory variables that provide substantive interpretations of the underlying nature of an X! Y relationship. Mediational designs have become ubiquitous in the organizational literature. Wood, Goodman, Cook, and Beckman (in press) reviewed five leading management journals over the years and identified 381 studies that tested mediational relationships. Of these, over 60 per cent of the studies followed prescriptions offered by Baron and Kenny (1986) or James and Brett (1984). However, the state of the art in mediational analysis is far from consistent. The fact that mediational designs have developed in different disciplines has only exacerbated the situation (Alwin & Hauser, 1975; Baron & Kenny, 1986; Frazier, Tix, & Barron, 2004; Holmbeck, 1997; James & Brett, 1984). Indeed, * Correspondence to: John E. Mathieu, University of Connecticut, 2100 Hillside road, Unit 1041, Storrs, CT , U.S.A. JMAthieu@business.uconn.edu y This article was published online on 7 September An error was subsequently identified and corrected by an Erratum notice that was published online only on 13 October 2006; DOI: /job.426. This printed version incorporates the amendments identified by the Erratum notice. Copyright # 2006 John Wiley & Sons, Ltd. Received 29 April 2006 Accepted 5 May 2006

2 1032 J. E. MATHIEU AND S. R. TAYLOR MacKinnon, Lockwood, Hoffman, West, and Sheets (2002) noted that Reflecting their diverse disciplinary origins, the procedures [for testing mediating variables] vary in their conceptual basis, the null hypothesis being tested, their assumptions, and statistical methods of estimation (p. 84). Mediational designs come in a variety of forms that differ in terms of the nature of the variables involved in this X! M! Y chain. For example, Judd and Kenny (1981) discussed the influence of psychological treatments on individuals behavior as mediated by their knowledge. Saks (1995) studied the influence of training on newcomer adjustments, as mediated by their post-training self-efficacy. Mayer and Gavin (2005) considered how employees trust in their management influenced their in-role performance and organizational citizenship behaviors, as mediated by their ability to focus their attention on work activities. Chen, Gully, Whiteman, and Kilcullen (2000) tested the influence of individuals psychological traits on their behavior, as mediated by their psychological states. Claessens, Eerde, Rutte, and Roe (2004) considered the influence of individuals planning behavior and work characteristics on their work outcomes such as strain, job satisfaction, and performance, as mediated by their perceived control of time. In all instances, mediational models advance an X! M! Y causal sequence, and seek to illustrate the mechanisms through which X and Yare related. However, there are important nuances in such designs that are often not appreciated, such as whether a mediator variable partially or full accounts for an X! Y relationship, or whether it merely serves as a linking mechanism between variables. This important distinction along with other aspects of mediational designs constitutes our focus. Given the diversity of approaches and statistical techniques that currently exist for testing mediation, our aims for this paper are to: (1) revisit the research design and measurement preconditions that must be met in order for tests of mediational relations to be meaningful; (2) review definitions of mediators and related concepts, and in so doing distinguish between indirect effects, and partial and full mediational models; (3) distinguish the different statistical tests and decision points that apply, depending on what type of relationship is hypothesized; and (4) provide an empirical example that illustrates such differences. We conclude with a discussion of directions for future research and theory incorporating these distinctions. We note at the onset that many of the points we make below have been voiced previously. However, a quick perusal of the literature will reveal that some conventional bits of wisdom have been routinely ignored by many authors, and there remains a lack of consensus regarding how mediational hypotheses should be framed and tested (Wood et al., in press). Our hope is that this presentation can serve as a guide for those wishing to advance and to test mediational type relations. Preconditions for Mediation Tests The basic mediational design tests whether some antecedent condition X has a relationship with some criterion Y through some intervening mechanism M. In other words, mediational design advance an X! M! Y style causal chain. Later we will distinguish different types of such designs and argue that they advance different a priori hypotheses. Nevertheless, the important point to emphasize here is that mediational designs implicitly depict a causal X! M! Y chain. Whereas substantial development has occurred surrounding statistical tests of mediated relationships, far less attention has been devoted to conditions for strong causal inference in such designs. We submit that inferences of mediation are founded first and foremost in terms of theory, research design, and the construct validity of measures employed, and second in terms of statistical evidence of relationships. The greatest challenges for deriving mediational inferences relates to the specification of causal order among variables, and the construct validity of the measures employed to operationalize X, M, and Y. In this sense, the

3 MEDIATIONAL INFERENCES 1033 preconditions for mediation tests are quite similar to those involved in specifying and testing causal (i.e., structural) models (see James, Mulaik, & Brett, 1982; Kenny, 1979). 1 Causal sequence Perhaps most fundamentally, inferences concerning mediational X! M! Y relationships hinge on the validity of the assertion that the relationships depicted unfold in that sequence (Stone-Romero & Rosopa, 2004). In other words, as with structural modeling techniques, multiple qualitatively different models can be fit equally well to the same covariance matrix. Using the exact same data, one could as easily confirm a Y! M! X mediational chain as one can an X! M! Y sequence (MacCallum, Wegener, Uchino, & Fabrigar, 1993; Stezl, 1986; Stone-Romero & Rosopa, 2004). Despite passionate pleas to the contrary by Mitchell (1985) and others, a clear trend away from the use of experimental designs and a parallel increase in correlational designs has been evident in organizational research in the past two decades (Scandura & Williams, 2000). Indeed, commenting upon that current state of the literature, Spencer, Zanna, and Fong (2005, p. 845) lamented that this [correlational mediational] analysis strategy is overused and has perhaps been elevated as the gold standard of tests of psychological processes and may even be seen in some quarters as the only legitimate way to examine them. In short, no statistical analysis can unequivocally differentiate one causal sequence from another. Theorists and researchers must then rely on other means to justify the sequence of effects. The most valuable bases to advance such inference come from: (1) experimental design features; (2) temporal precedence; and (3) theoretical rationales. Experimental designs Naturally, experimental designs afford the strongest foundation for making causal inferences. Hallmarks of randomized experimental designs include random assignment of participants to conditions, control of extraneous variables, and experimenter control of the independent variable. Indeed, the philosophy of experimental designs is to isolate and test, as best as possible, X! Y relationships from competing sources of influence. In mediational designs, however, this focus is extended to a three phase X! M! Y causal sequence. The benefits of conducting randomized experiments for testing such sequences has long been recognized. For example, Baron and Kenny (1986) described a design (based on Smith, 1982) whereby one introduces two experimental manipulations: (1) one presumed to influence the mediator and not the criterion; and (2) one presumed to influence the criterion yet not the mediator. Analyses of such designs would permit one to distinguish factors that exert influence directly on a criterion versus those that are carried through an intervening mechanism. More recently, Stone-Romero and Rosopa (2004, p. 283) argued that the only way that one can make credible inferences about mediation is to perform two or more experiments. In the first, the cause [i.e., X] is manipulated to determine its effect on the mediator [i.e., M]. In the second, the mediator [i.e., M] is manipulated to determine its effect on the dependent variable [i.e., Y]. Certainly such an approach affords a solid foundation for making causal inferences, but may not be feasible or even desirable in many applied circumstances. There may well be ethical, logistical, financial, and other considerations that limit the extent to which researchers can employ randomized experimental designs. 1 We should further note that mediation inferences from such designs are predicated not only on the assumptions that X, M, and Y are causally ordered in that fashion, and their relationships are not attributable to other variables or processes, but also that they are related linearly (see Bollen, 1989; Kenny et al., 1998; Pearl, 2000 for further details). Whereas non-linear relationships, such as moderation, can be incorporated in mediational frameworks, that takes us beyond the current discussion (see Baron & Kenny, 1986; James & Brett, 1984; Muller et al., 2005).

4 1034 J. E. MATHIEU AND S. R. TAYLOR Spencer et al. (2005) outlined circumstances when experiments offer desirable features for drawing mediational inferences, as well as instances when they may be less applicable. In short, however, randomized experimental designs offer the strongest basis for drawing causal inferences and should not be abandoned so prematurely by applied researchers (Mitchell, 1985; Scandura & Williams, 2000). Quasi-experimental designs also afford means by which causal order can be established (Cook & Campbell, 1979; Shadish, Cook, & Campbell, 2002). Because researchers may not be able to randomly assign participants to conditions, the causal sequence of X! M! Y is vulnerable to any selection related threats to internal validity (Cook & Campbell, 1979; Shadish et al., 2002). To the extent that individuals status on a mediator or criterion variable may alter their likelihood of experiencing a treatment, the implied causal sequence may also be compromised. For example, consider a typical: training! self-efficacy! performance, mediational chain. If participation in training is voluntary, and more efficacious people are more likely to seek training, then the true sequence of events may well be self-efficacy! training! performance. If higher performing employees develop greater self-efficacy (Bandura, 1986), then the sequence could actually be performance! efficacy! training. If efficacy and performance levels remain fairly stable over time, one could easily misconstrue and find substantial support for the training! efficacy! performance sequence when the very reverse is actually occurring. Researchers also have less control over contaminating variables in quasi-experiments as compared to randomized experiments. Whereas concerns about contaminating influences and other threats to internal validity are extensive and well discussed elsewhere (see Cook & Campbell, 1979; Shadish et al., 2002; West, Biesanz, & Pitts, 2000), our primary focus here concerns threats to the implied causal sequence of effects in a mediational design. Revisiting our training example, a misspecification of causal sequence can emanate from the influence of an omitted (sometimes referred to as unmeasured, third, contaminating, hidden, or confounding) variable. For example, if employees with greater seniority are first eligible to receive training, and if they also tend to have higher self-efficacy, then there would be an illusionary training! efficacy relationship unless seniority is also controlled. The issue here is that one must carefully consider what other variables might confound the relationships under consideration and account for their influence when evaluating the specified causal sequence and variable relationships as outline above. Otherwise, such influences might mask real effects (see MacKinnon, Krull, & Lockwood, 2000) or generate artifactual relationships. In summary, our discussion about the advantages of experimental and quasi-experimental designs converges on the larger issues of justifying the presumed causal order of variables and minimizing the influence of unmeasured variables. Randomized experiments certainly provide the strongest case for minimizing the influence of such potential effects but are difficult to implement. Quasi-experiments offer much as well, but are susceptible to a variety of threats to causal inferences. James (1980) and James et al. (1982) have well chronicled this issue. They submitted that an omitted variable poses a threat to causal inferences if it: (a) has a significant unique influence on an effect (i.e., mediator or criterion); (b) is stable; and (c) is related to at least one other predictor included in the model. In other words, in mediation analyses, omitted variables represent a significant threat to validity of the X! M relationship if they are related both to the antecedent and to the mediator, and have a unique influence on the mediator. Moreover, omitted variables represent a significant threat to inferences involving the prediction of the criterion, if they have a unique influence on the outcome variable and either the antecedent or mediator. Temporal precedence A mediational framework proposes that the antecedent preceded the mediator, which in turn preceded the criterion. Implicitly, therefore, mediational designs advance a time-based model of events whereby

5 MEDIATIONAL INFERENCES 1035 X occurs before M which in turn occurs before Y. To the extent that the measures and operations employed to operationalize variables in a study are aligned with that sequence, one can have more confidence that the chain of relationships is not compromised. Let us emphasize one point here: it is the temporal relationships of the underlying phenomena that are at issue, not necessarily the timing of measurements. Certainly, to the extent that the two different sequences are aligned is of concern. However, the literature is replete with designs whereby researchers collect a set of observations and then correlate variables with some record of last year s performance, whether that was derived from performance appraisals, performance outputs, sales, or some financial index. In effect, this reverses the presumed sequence of events and more likely models Y! X! M than it does X! M! Y. Simply assessing a presumed antecedent before measuring a presumed mediator and criterion in no way assures that the true underlying causal order is consistent with the order of measurement (James et al., 1982; Kenny, 1979). Indeed the synchronization of measurement timing and the development of phenomena over time are critical to the basis of causal inferences (Mitchell & James, 2001). Given that X! M! Y relationships are presumed to unfold over time, it begs the question of how long does it take each variable to develop and to change? Consider a work redesign effort intended to empower employees and thereby to enhance their work motivation with the aim of increasing customer satisfaction. How long does it take to establish the new work design? Over what duration should we track employees subsequent motivation? If employees are indeed more motivated to perform, how long will it take for customers to notice and for them to become more satisfied? These questions are not easy to answer, and in few instances would phenomena readily align with the 3 or 6 month intervals that organizations are willing to tolerate, even if they are open to multiple data collections. Even worse, consider the fact that employee motivations (M) in this instance are likely to begin changing before the work redesign (X) intervention is fully entrenched. And, the appropriate window for sampling customer reactions may vary widely depending on their frequency of encounters with employees and other factors. In sum, the guiding point here is that the passage of time between the assessment of X, M, and Y helps to further strengthen inferences about the causal sequence. To the extent that such assessments are aligned with the underlying developmental phenomena being studied will strengthen causal inferences. Theoretical guidance Theoretical frameworks usually prescribe a distinct ordering of variables. In fact, it is a hallmark of good theories that they articulate the how and why variables are ordered in a particular way (e.g., Sutton & Staw, 1995; Whetten, 1989). This is perhaps the only basis for advancing a particular causal order in non-experimental studies with simultaneous measurement of the antecedent, mediator, and criterion variables (i.e., classic cross-sectional designs). For example, Fishbein and Aijzen s (1975) Theory of Reasoned Action has long posited that individuals attitudes give rise to intentions, which in turn influence their actual behaviors. This theoretical foundation has been applied extensively to the study of employees absence and turnover behaviors (Hom & Kinicki, 2001; Tett & Meyer, 1993). The job characteristics model argues that work design features give rise to psychological states which in turn influence individuals reactions (e.g., satisfaction) and behaviors (Hackman & Oldham, 1980). Mathieu (1991) used Lewinian Field Theory (Lewin, 1943) to submit that variables more psychologically removed from oneself (i.e., distal effects such as perceptions of work characteristics), would influence more psychologically proximal variables (e.g., role states) and thereby affect work attitudes (e.g., satisfaction, organizational commitment). Absent an experimental or longitudinal design, one might test a mediational model on the basis of the theoretical ordering of variables. Naturally the case would be stronger if one could also leverage features from a design perspective, but clearly the theory must articulate a certain causal sequence. And, in cross-sectional studies one often has little else to justify any particular order.

6 1036 J. E. MATHIEU AND S. R. TAYLOR To summarize, the specification of the causal order of variables is absolutely critical to inferences about mediational relationships. This is first and foremost a theoretical exercise. Research design features in terms of experimental control and temporal precedence provide additional justification for particular sequences. Notably, there is no panacea for justifying causal sequence. Learned scholars differ on what they believe is sufficient grounds upon which to claim causal order. On one extreme, Stone-Romero and Rosopa (2004) submitted that anything short of a randomized experiment is insufficient to claim justified causal order. Their position is tests of mediation models that are based upon data from non-experimental studies have little or no capacity to serve as a basis for valid inferences about mediation (Stone-Romero & Rosopa, 2004; p. 250). On the other extreme, a perusal of journal articles will quickly reveal numerous instances of authors claiming causal connections from mediational analyses of cross-sectional data collected in a single survey as related to last year s performance indices. Reasonable people can disagree, and we personally believe that both of the above extreme positions are probably overstated. In any case, to the extent that one s work is: (a) grounded in strong theory; (b) employs true or quasi-experimental designs; and (c) assesses variables over time in the proper sequence and intervals, confidence in the causal sequence of variables in a particular model is enhanced. Measurement related issues As with any research investigation, the construct validity of measures employed are of concern in tests of mediation. Schwab (1980) submitted Construct validity is defined as representing the correspondence between a construct (conceptual definition of a variable) and the operational procedure to measure or manipulate that construct (pp. 5 6). Of note in particular for mediational analyses, attention should be directed at the convergent and discriminant validity of measures. Convergent validity Convergent validity essentially concerns the extent to which different measures of the same construct hold together or converge on the intended construct. Usually convergent validity is assessed using techniques such as factor analyses and other approaches that evaluate how well different observations relate to a latent variable. Naturally, this concept is related to reliability concepts such as internal consistency estimates, alternative forms/methods, interrater, or test-retest. Depending on the nature of the constructs involved in the X! M! Y relationship, any combination of reliability estimates may be applicable (see Nunnally, 1978). Of note for the present discussion is the fact that measurement unreliability, particularly that of the mediator, can bias mediational analyses. As Hoyle and Kenny (1999) have demonstrated, assuming all positive paths, to the extent that a mediator is measured with less than perfect reliability, the M! Y relationship would likely be underestimated, whereas the X! Y would likely be overestimated when the antecedent and mediator are considered simultaneously. Whereas latent variable modeling can help to compensate for measurement shortcomings, the technique is certainly not a panacea. Consequently, the message here is clear: it is critical to use reliable measures when testing mediation, particularly when it comes to the mediator variable. Discriminant validity Discriminant validity of measures is another concern for all research investigations, yet particularly for tests of mediation. Discriminant validity refers to the extent to which measures of different constructs are empirically and theoretically distinguishable. Note that discriminant validity must be gauged in the context of the larger nomological network within which the relationships being considered are believed to reside. Discriminant validity does not imply that measures of different constructs are uncorrelated;

7 MEDIATIONAL INFERENCES 1037 indeed if that were the case there would be no mediational covariance to be modeled. The issue is whether measures of different variables are so highly correlated as to raise questions about whether they are assessing different constructs. If measures of an antecedent variable and a mediator are not sufficiently distinguishable, then they are in effect tapping the same underlying domain. Consequently, any attempt to parse their independent contributions to a criterion variable will be futile. For example, if X and M fail to evidence discriminant validity, any sequential analysis of their substantive relationship will conclude that the mediator carries the influence of X on Y. The same conclusion would follow in situations where the mediator and criterion and not distinguishable. This problem is akin to the notion of multi-collinearity between either the X and M variables, or between the M and Y variables. Consequently, it is incumbent on researchers to demonstrate that their measures of X, M, and Y evidence acceptable discriminant validity before any mediational tests are justified. This may be done in a variety of fashions ranging from exploratory factor analyses to more powerful multi-trait, multi-method approaches, and confirmatory factor analyses. In sum, a lack of discriminant validity between either X and M, or between M and Y, will lead to an illusionary mediational relationship that amounts to nothing more than correlating some measure of a construct with another measure of the same construct. Distinguishing indirect and mediating relationships Up to this point we have employed the term mediating variable in a very general sense. Unfortunately, different authors define mediation in many different ways and often use terms such as indirect effects, intervening variables, intermediate endpoint, and so forth interchangeably with mediators. (see MacKinnon et al., 2002 for a review). Further, although mediational models are pervasive in applied research and elsewhere, there is some debate concerning the requisite statistical evidence for drawing inferences about mediation (cf., Baron & Kenny, 1986; Collins, Graham, & Flaherty, 1998; Frazier et al, 2004; Holmbeck, 1997; James & Brett, 1984; James, Mulaik, & Brett, 2006; Kenny, Kashy, & Bolger, 1998; MacKinnon, et al., 2000, 2002; Preacher & Hayes, 2004; Shrout & Bolger, 2002). We believe that root causes of such controversies lie in differences of opinion regarding: (1) definitions of mediators and related concepts; (2) the necessity of first demonstrating a significant total X! Y relationship; and (3) the appropriate base model for tests of different forms of mediation. The first two points of contention are closely intertwined. Some have submitted that a precondition for tests of mediation is that the antecedent must exhibit a significant total relationship with a criterion when considered alone (i.e., X! Y, see Baron & Kenny, 1986; Judd & Kenny, 1981; Preacher & Hayes, 2004). Others have relaxed this precondition, and argued that mediation inferences are justified if the indirect effect carried by the X! M and M! Y paths is significant (e.g., Kenny, et al., 1998; MacKinnon et al., 2002). Advocates of this latter view often equate mediator variables with indirect effects (e.g., Alwin & Hauser, 1975; Bollen, 1987; MacKinnon et al., 2002). However, there is an important distinction between indirect and mediator variables. For example, MacKinnon et al. (2002, p. 83) suggested that An intervening variable (Mediator) transmits the effect of an independent variable to a dependent variable [emphasis added]. In contrast, Baron and Kenny (1986, p. 176) submitted that a given variable may be said to function as a mediator to the extent that it accounts for the relation between the predictor and the criterion [emphasis added]. Preacher and Hayes (2004, p.719) explicitly drew a distinction between the two concepts and argued that mediation is a special, more restrictive, type of intervening relationship. A conclusion that a mediation effect is present implies that the total effect X! Y was present initially. There is no such assumption in the assessment of indirect effects. It is quite possible to find

8 1038 J. E. MATHIEU AND S. R. TAYLOR that an indirect effect is significant even when there is no evidence for a significant total effect. Whether or not the effect also represents mediation should be judged through examination of the total effect. In other words, mediator variables are explanatory mechanisms that shed light on the nature of the relationship that exists between two variables. If no such relationship exists, then there is nothing to be mediated. While a chain of events whereby X! M and M! Y may well be of interest, along with the extent to which variance in Y can be attributed to the indirect effect of X, we submit that sequence represents a qualitatively different phenomenon than mediation. We prefer to label such relationships as indirect effects. We readily acknowledge that there is another reason why some researchers (e.g., MacKinnon et al., 2000) advocate dropping the X! Y precondition for mediational inferences. This second position centers around the fact that confounding, suppression, and interactive effects could attenuate overall X! Y relationships (see MacKinnon et al., 2000). 2 Similarly, others have argued that competing effects might mitigate total X! Y relationships, when opposite signed direct and indirect effects are present (e.g., when X and M are both positively related to Y, yet X and M are negatively related). Notably, the common thread through all of these positions is that some other variable, including perhaps the mediator, serves to contaminate the total X! Y relationship when viewed in isolation. In other words, the opposite signs and mediator as a suppressor arguments both suggest that the true underlying model is a partially mediated one whereby the direct effect of X! Y can only be interpreted in the context of a model that also includes the M! Y path. This raises a central point about the importance of the base model that one hypothesizes. James et al. (2006) underscored the importance of the base model that one adopts for tests of mediation. In the case of full mediation, M is hypothesized to fully account for the significant total effect of X! Y. In other words, the direct effect of X! Y is no longer significant once M! Y has been included. In contrast, in the case of partial mediation, M is believed to account for a significant portion of the total X! Y, but a significant direct effect also remains. In other words, both M! Y and X! Y are significant when considered simultaneously. Of course, both partial and full mediation models are predicated on a significant X! M relationship. James et al. (2006) and Shrout and Bolger (2002) noted that full and partial mediational inferences rely on different types of statistical tests. Consequently, which model one hypothesizes may lead to different conclusions in many instances. James et al. (2006) noted that the Baron and Kenny (1986) approach implicitly advocates partial mediation as the base model for tests of mediation. Alternatively, James and Brett (1984), and James et al. (2006) prefer the axiom of parsimony and advocate the full mediation base model. All agreed that substantive reasoning should guide which is adopted in any given circumstance; but the important point is that the a priori model that one advances has important implications for confirmatory and disconfirming statistical evidence. We extend this logic and submit that the specification of one s hypothesized base model has implications for indirect effects along with partial and full mediation. Moreover, such relations can be examined in the context of larger structural models where the influences of other variables of interest are also considered. Nevertheless, the evidential basis for drawing inferences of each type remains consistent and is outlined below. In summary, we believe that there are different types of relationships that fall under the general heading of intervening effects. Accordingly, we use the term intervening effects to describe any type of 2 Concerns about the influence of interactive or confounding variables imply the presence of non-linear relationships which violates an assumption of testing indirect or mediated relations, unless one is also hypothesizing moderation (see Footnote 1). Further, extraneous, omitted, or 3rd variables represent specification errors that always must be accounted for, through theoretical, methodological, or empirical means, whenever a causal sequence of effects is advanced (James et al., 1982; Stone-Romero & Rosopa, 2004). We elaborate more fully on this and related points below.

9 MEDIATIONAL INFERENCES 1039 linking mechanism M that ties an antecedent with a criterion. Indirect effects are a special form of intervening effect whereby X and Y are not related directly (i.e., are uncorrelated), but they are indirectly related through significant relationships with a linking mechanism. In contrast, mediation refers to instances where the significant total relationship that exists between an antecedent and a criterion, is accounted for in part (partial mediation) or completely (full mediation) by a mediator variable. Estimation Guidelines and Decision Rules We submit that different statistical rules of evidence apply depending on whether one anticipates an indirect effect versus partial or full mediation. We argue that researchers are obliged to specify, a priori, which type of intervening process that they anticipate. Importantly, the nature of the hypothesized relationship leads to different sources of confirmatory and disconfirming evidence. In this sense, what we are advocating is an approach that is similar to that of structural equation modeling (SEM). Accordingly, a failure to reject a hypothesized model hinges on two types of tests: (1) confirmation of hypothesized relations (i.e., relationships that were hypothesized to exist are indeed significant and in the hypothesized directions); and (2) disconfirmation of non-hypothesized paths (i.e., sufficient model fit indices, which indicate that the paths that were hypothesized to be absent are indeed not significant). Moreover, because different competing models can be fit to the same data, we advocate contrasting one s hypothesized model against viable alternative models (see Anderson & Gerbing, 1988, for a good background on this general approach). Figure 1 presents three alternative models containing intervening effects and their respective parameters. In this sense, the indirect effects model is the most constrained or parsimonious, as it implies that the only significant relationships observed are the combined effect (b mx b ym ). This implies that both the X! M(b mx ) and M! Y(b ym ) paths are significant, although the combined effect is best tested using approaches such as the Sobel test (see MacKinnon et al., 2002; Shrout & Bolger, Indirect Effect X β mx M β ym Y Full Mediation X β mx M β ym Y β yx Partial Mediation X β mx M β ym. x Y β yx. m Figure 1. Alternative intervening models

10 1040 J. E. MATHIEU AND S. R. TAYLOR 2002 for details). Importantly, an indirect effect hypothesis also implicitly suggests that the total X! Y relationship (b yx ) is absent. The full mediation model is the next most parsimonious. This model also includes significant X! M(b mx ) and M! Y(b ym ) paths. However, the dashed line from X! Y in this model is meant to imply a significant total X! Y(b yx ) relationship that becomes non-significant when M! Y(b ym ) is included. In other words, a hypothesis of full mediation also requires a non-significant b yx.m effect. Last, the partial mediation model is the least parsimonious and implies that X! M(b mx ), as well as both M! Yand X! Y will be significant when considered simultaneously (b ym.x and b yx.m, respectively). The three panels of Figure 2 specify sequences of effects to be considered in order for each of the hypothesized models to be supported. Later we will describe analytic techniques which provide information regarding these effects. The columns of rectangles in Figure 2 depicts the various conditions that must hold for each model to be accepted, whereas the branches containing circles depict guided alternative hypotheses that might be considered if a hypothesized condition is disconfirmed. Notably, by accepted models we mean that the data fail to reject the hypothesized model. As is always the case, this does not mean that a hypothesized model has been proven; simply that it is not inconsistent with the data. Similarly, once any facet of a hypothesized model is rejected, one enters an exploratory mode as alternative models are considered. Consequently, any conclusions that are derived from such searches are tentative at best and need to be validated on a new sample. Indirect effects As shown in Panel 1 of Figure 2, the pivotal test of the indirect model is simply (b mx b ym ) using methods such as the Sobel (1982) test or more sophisticated approaches employing bootstrapping techniques (see, MacKinnon et al., 2002; Preacher & Hayes, 2004; Shrout & Bolger, 2002). If such a test is not significant, then one should reject the indirect effect hypothesis and consider viable alternatives. A more parsimonious alternative in such instances would be to consider simply a direct Figure 2. Decision tree for evidence supporting different intervening effects

11 MEDIATIONAL INFERENCES 1041 X! Y(b yx ) relationship. Moreover, even if the indirect effect was significant, researchers should consider whether alternative models such as a partially or fully mediated model are suggested by the data. For example, if the overall X! Y relationship was significant (b yx ), and X does not contribute to the prediction of Yonce M has been considered (b yx.m ), then the hypothesis of an indirect effect would be rejected in lieu of an alternative model of full mediation. This approach echoes our earlier comments about researchers casual dismissal of the precondition of a total X! Y effect for tests of mediation. Our reading of the literature suggests that there are very few instances where researchers actually hypothesized a priori that the total X! Y relationship would be non-significant. Rather, it appears as though many evoke a waiver of the X! Y precondition when it fails to materialize in their data, and then simply proceed to test the significance of the indirect effect. This has occurred even when there was no evidence to suggest suppression or counter-acting signs of direct and indirect effects. Such tactics, in our opinion, moves one away from confirmatory hypothesis testing and into the exploratory realm. In summary, we submit that the presence of a significant total X! Y relationship leads to the rejection of a hypothesis of an indirect effect and should trigger a consideration of an alternative partial mediation hypothesis (if suppression or counter-acting effects are suspected), and thereby perhaps, to a full mediation explanation. Full mediation As depicted in the second panel of Figure 2, a hypothesis of full mediation is predicated on a significant total X! Y(b yx ) relationship. Failing that, one might consider an alternative hypothesis of an indirect effect. If suppression is evident, then one might consider the alternative hypothesis of a partially mediated relationship. Assuming the total effect was present, one proceeds to test the X! M(b mx ) and M! Y(b ym ) relationships. If either fails to exist, then the evidence is consistent with the alternative hypothesis of a direct effect. Moreover, full mediation depends on the non-significance of direct effect of X! Y when the M! Y path is included (i.e., a non-significant b yx.m ). If the direct X! Y path is significant in this context, then the hypothesis of full mediation should be rejected and the researcher should consider the alternative hypothesis of partial mediation. We should note that there may be cases where adding the b yx.m parameter attenuates the M! Y relationship (b ym.x ) to a non-significant level. If the X! Y relationship (b yx.m ) is significant in such an instance, then the full mediational hypothesis should be rejected in lieu of an alternative hypothesis of a direct effect. Alternatively, if neither b yx.m or b ym.x are significant, and the previous conditions were satisfied, then the data are consistent with the hypothesis of full mediation. This follows from the fact that the relevant M! Y parameter for the full mediation hypothesis is b ym not b ym.x (see James et al., 2006). Partial mediation A partial mediation hypothesis, as shown in Panel 3 of Figure 1, is the least constrained and rests on the significance of all three paths: X! M(b mx ) and both X! Y(b yx.m ) and M! Y(b ym.x ) when considered simultaneously. Given the presumed causal order of variables, if the X! Y(b yx.m ) path is not significant in this model, then the hypothesis of partial mediation should be rejected and one should, perhaps, consider an alternative hypothesis of full mediation. Alternatively, if the X! M(b mx ) or the M! Y(b ym.x ) paths are not significant, then the partial mediation hypothesis should be rejected in lieu of the alternative hypothesis of simply a direct effect. The partial mediation hypothesis would only be supported if all three hypothesized paths are significant.

12 1042 J. E. MATHIEU AND S. R. TAYLOR Caveats We should highlight two other related concerns for the tests outlined above. First, our various decision points pivot on tests of statistical significance, just as any SEM nested model comparisons do (see Anderson & Gerbing, 1988). Nevertheless, tests of statistical significance must be considered in light of related issues such as sample size and measurement reliability (Hoyle & Kenny, 1999). In other words, significance tests must be tempered by considerations of power and effect size estimates. Enormous sample sizes can yield statistically significant results that are virtually meaningless in practice and also easily lead to the rejection of full mediation hypotheses. Alternatively, small sample sizes can easily lead to inferences of full mediation in instances where there is not sufficient power to adequately test for partial mediation. The summary point is that sufficient power must exist to adequately test various relationships, and researchers should balance conclusions about statistical significance with those about practical significance. A second, and related caveat, concerns the relative power of different tests. MacKinnon et al. (2002) have argued that a direct test of intervening effects (b mx b ym ) has greater power as compared to causal steps approaches such as those outlined by Baron and Kenny (1986). However, recall that MacKinnon et al. (2002) equated indirect effects with mediation, and argued that overall, the step requiring a significant total effect of X on Y led to the most Type II errors, (p. 96). In other words, the distinguishing feature between indirect and mediator relations is what accounts for the fact that tests of the latter are more conservative than the former. The fact that mediators rely on the presence of a total direct effect represents a greater statistical burden, however, so we believe that the corresponding lower power is totally appropriate. In other words, the combined tests of indirect effects appear to have greater statistical power simply as a consequence of comparing them with qualitatively different types of relationships namely, mediation. Summary Tests of intervening effects are predicated on the assumption that the causal sequence of variables is sufficiently justified and the measures employed to represent the constructs possess sufficient construct validity. While not particularly controversial, we believe these preconditions are often overlooked and should be afforded more attention by scholars. Less clarity surrounds the rules of evidence for mediational type inferences and associated statistical tests. A key to most of this confusion is the fact that X! M! Y models may represent full mediation, partial mediation, or indirect effects all of which are confirmed or disconfirmed in slightly different ways. We submit that researchers are obliged to a priori specify the nature of the relationship(s) that they anticipate, and then to conduct the corresponding tests to demonstrate both confirmatory and disconfirming evidence. To better illustrate how this works in practice, we offer the following example. Empirical Illustration The purpose of this illustration is to demonstrate the steps and evidential basis involved in testing indirect effects, and partial and fully mediated relationships. Our example focuses on the concept of

13 MEDIATIONAL INFERENCES 1043 Figure 3. Hypothesized intervening effects self-efficacy as a mediator of the influences of individual differences and situational cues on individuals performance and is illustrated in Figure 3. Self-efficacy is defined as people s judgments of their capabilities to organize and execute courses of action required to attain designated types of performances (Bandura, 1986, p. 391). The positive relationship between self-efficacy and performance has been demonstrated time and again and summarized in narrative reviews (e.g., Bandura & Locke, 2003) and in meta-analyses (e.g., Stajkovic & Luthans, 1998). Hence, there is abundant support for viewing it as an influence on performance. Bandura (1977) has long theorized that self-efficacy is determined mostly by the cognitive appraisal and integration of information cues. Two of the most influential of such cues are enactive mastery and vicarious experience. Enactive mastery develops through repeated performance accomplishments in the same or similar situations. In other words, to the extent that individuals have performed well in a particular situation in the past, it is reasonable to expect that their efficacy expectations will be higher. Further, naturally one would expect that individuals past performance would correlate positively with their future performance for reasons other than simply their efficacy expectations (e.g., because ability influences both). Consequently, as depicted in Figure 3, we would hypothesize that self-efficacy would partially mediate the relationship between previous (i.e., baseline) performance and subsequent performance. Vicarious experience is gained through direct observation or information about how well others have performed in a situation. However, there is no reason to expect that vicarious experience would influence individuals performance unless they internalized such information in terms of their efficacy expectations. Consequently, we hypothesized that self-efficacy would fully mediate the relationship between normative information and individuals performance. Indeed, previous research has been consistent with this expectation (e.g., Mathieu & Button, 1992; Weiss, Suckow, & Rakestraw, 1999). Finally, recent theorizing and research have argued that relatively stable individual differences may influence efficacy expectations. For example, Phillips and Gully (1997) found support for a positive correlation between individuals learning goal orientation and their self-efficacy in an academic setting. Individuals who are high on learning goal-orientation strive to understand something new or to increase their level of competence in a given activity (Button, Mathieu, & Zajac, 1996). Whereas a learning goal orientation may contribute directly to performance, it is more likely to help shape specific task-related perceptions such as self-efficacy. Although some previous researchers have found results that are consistent with a hypothesis of full mediation (e.g., Phillips & Gully, 1997), others have found relationships more consistent with an indirect effect inference (e.g., Chen et al., 2000; Diefendorff, 2004; Potosky & Ramakrishna, 2002). Which interpretation is most appropriate is debatable. However, for present illustration purposes, we hypothesized that learning goal orientation would exhibit a positive indirect effect with performance via self-efficacy.

14 1044 J. E. MATHIEU AND S. R. TAYLOR Method Participants Two hundred and one undergraduates were recruited from introductory psychology courses at a large northeastern University and received extra credit toward their course grade for participation. The sample was 61 per cent female, and their average age was (SD ¼ 1.97). Participants were invited to attend experimental sessions where they were randomly assigned to one of the three normative information conditions (n ¼ 67 per condition) as described below. Task and procedure The task was identical to the one used by Mathieu and Button (1992). It involved the creation of words containing three or more letters drawn from a list of 10 letters: four vowels (worth 1 point each), two consonants (worth 1 point each), and four additional consonants worth 2, 3, 4, and 10 points, respectively. The point values associated with each letter correspond to those used in the game Scrabble. The object of the task was to score as many points as possible during a 10-minutes session by forming words containing three or more letters from the list provided, excluding proper nouns and slang. Points were awarded according to the point values associated with letters used in each word generated. Upon arrival at the experimental session, participants completed an informed consent form and a survey that contained demographic items and a measure of learning goal orientation from Button et al. (1996). Once the survey was completed, the experimental task was explained to participants and they completed a 5-minutes practice exercise. They then calculated their own score on the practice exercise and were told that they would perform a 10-minutes experimental trial after answering some survey questions. The first page of the survey instrument presented the normative information manipulation using the following statement. In previous testing we have found that students like you score about 115 [175, 235] points on the task you are about to complete [emphasis in instructions]. The middle value corresponded to pilot subjects average performance, whereas the low- and high-point values were set one standard deviation from the mean, based on pilot data. They then completed several survey items that included their self-efficacy and a manipulation check, and then completed the 10-minutes task, were debriefed and given their extra credit slips, and dismissed. Measures Manipulation check A manipulation check identical to that used by Mathieu and Button (1992) was administered after the survey items. It asked participants how many points they thought most people would score on the task they were about to complete. As anticipated, their responses differed significantly across the normative information conditions (F(2,194) ¼ , p < 0.001) and all means differed significantly from each other in the anticipated fashion. Performance Participants practice and task performances were simply the total number of points they earned during each of the timed periods. Individuals scored their own practice trail in order to provide clear and