Comparative hypothesis testing via process tracing

Transcription

1 Comparative hypothesis testing via process tracing Keywords: causal inference; causal mechanisms; certainty; uniqueness; sufficiency Abstract: Causal inference via process tracing has received increasing attention during recent years. A 2x2 typology of hypothesis tests takes a central place in this debate. A discussion of the typology demonstrates that its role for causal inference can be improved further in three respects. First, the aim of this paper is to formulate case selection principles for each of the four tests. Second, in focusing on the dimension of uniqueness of the 2x2 typology, I show that it is important to distinguish between theoretical and empirical uniqueness when choosing cases and generating inferences via process tracing. Third, I demonstrate that the standard reading of the so-called doubly decisive test is misleading. It conflates unique implications of a hypothesis with contradictory implications between one hypothesis and another. In order to remedy the current ambiguity of the dimension of uniqueness, I propose an expanded typology of hypothesis tests that is constituted by three dimensions. Ingo Rohlfing Cologne Graduate School in Management, Economics and Social Sciences University of Cologne Richard-Strauß-Str. 2 D Köln rohlfing@wiso.uni-koeln.de Phone: Fax: A previous version of this paper was presented at the workshop Process tracing: philosophy, theory, and practice at the ECPR Joint Sessions of Workshops 2012 in Antwerp. I would like to thank the participants of the workshop for their comments. I am grateful to Chris Winship and comments by two anonymous reviewers. Valuable research assistance was provided by Nancy Deyo. The usual disclaimer applies.

2 1. Introduction Causal inference via process tracing has received increasing attention during recent years. A 2x2 typology of hypothesis tests, developed by van Evera (1997) in a general discussion of social science methods, and rediscovered by the recent literature on process tracing (Bennett 2008; 2010; Collier 2011; Mahoney 2012), takes a central place in this debate. The typology is constituted by two dimensions related to the observable implications that one can derive from a hypothesis. The first dimension is called certitude or certainty and captures how likely it is to gather a specific observable implication in process tracing. The second dimension is called uniqueness and asks whether an observable implication can be derived from a single or multiple hypotheses. 1 In a categorical view, the intersection of high and low certitude with uniqueness and non-uniqueness yields four tests that allow the derivation of inferences about a working hypothesis and rival hypotheses, conditional on whether the working hypothesis passes or fails the test (Bennett 2010; Collier 2011). The 2x2 typology adds valuable insight and rigor to hypothesis testing via process tracing. However, a review and discussion of the typology in this paper demonstrates that its role for causal inference can be improved further in three respects. First, there has not been a comprehensive discussion as to whether and how case study researchers can choose between the four tests. This is an important matter because each of the tests has different implications for causal inferences. Empirical researchers need to be equipped with knowledge about how to craft their design in order to realize one type of test or the other. One major aim of this paper is to formulate case selection principles for each of the four tests. Case selection related to the dimension of certainty has received some attention in the past (Bennett 2008), while there has been virtually no explicit reflection on the dimension of 1

3 uniqueness. 2 In the discussion of case selection, particular attention is directed to two salient problems with the current interpretation of testing unique implications. The solution of both problems represents the second and third way in which this paper seeks to improve the typology. Second, it is essential to distinguish between the theoretical uniqueness and empirical uniqueness of an observable implication. Theoretical uniqueness is present when there is only one hypothesis that yields a given prediction in principle. Empirical uniqueness is present when we can only test one hypothesis on the basis of a specific case. If this requirement is met, it is evident that empirical uniqueness is sufficient for generating an unambiguous inference for a hypothesis. However, I demonstrate that empirical uniqueness is not necessarily the best criterion to pursue. For some constellations involving a working hypothesis and a rival proposition, it is only possible to produce inferences on both when the selected case lacks empirical uniqueness. Consequently, establishing empirical uniqueness in such process-tracing studies unnecessarily constrains the range of inferences we can make and effectively limits causal inference to the working hypothesis. Furthermore, I show that failure to distinguish between the two forms of uniqueness is unfortunate because there is no unequivocal link between them. The consideration of empirical uniqueness alone is deficient because the test of an empirically unique implication does not convey all relevant information for causal inference on the hypotheses under scrutiny. On the other hand, an exclusive focus on theoretical uniqueness is inappropriate because it does not automatically ensure unambiguous causal inferences in the empirical analysis. Theoretical and empirical uniqueness should always go hand in hand and systematic case selection is the instrument linking the two in ways described in this paper. The variety of uniqueness-related constellations that one can confront is systematized by clarifying whether two hypotheses center on exactly the same cause, two exclusive causes, only 2

4 one of which can be present in a single case (country is rich or low, person male or female, etc.), or two non-exclusive causes (country is rich and democratic, person is male and old, etc.). The same distinction is made for the hypothesized outcome and the underlying mechanism. The distinction between identical, exclusive, and non-exclusive causes, outcomes, and mechanisms highlights the variety of designs in comparative hypothesis testing and the complex interplay between theoretical uniqueness, empirical uniqueness, and the generation of causal inferences. Building on this systematization, the third direction in which the typology is improved is specifically related to the so-called doubly decisive test. The literature attaches strong implications to the doubly decisive test combining high certitude with uniqueness. In the standard reading of the test, confirmation of one hypothesis automatically disconfirms the competing propositions (Bennett 2010: 211; Collier 2011: 825). A close examination of the dimension of uniqueness shows that this reading conflates the uniqueness of a proposition with mutually exclusive implications made by a working hypothesis and a competing hypothesis. The elaboration of multiple designs in which a working hypothesis and competing hypothesis can be involved highlights that unique implications do not necessarily entail contradictory implications. Based on the insight that the doubly decisive test intermingles two separate dimensions, I propose an expanded typology of hypothesis tests that is constituted by three dimensions and enhances case selection and causal inference: certainty, uniqueness, and mutual exclusiveness. Although a 2x2x2 typology is not as handsome as a 2x2 typology, the expansion is necessary in order to remove the current ambiguity inherent in the latter. Throughout the paper, principled methodological arguments are illustrated with empirical examples from various fields of research in the social sciences (without the intention to make any substantive contribution to any of the studies that is addressed). 3

5 2. The 2x2 typology of hypotheses tests The typology of hypothesis tests recently rediscovered in the case study literature dates back to van Evera s (1997: 30-32) distinction between the uniqueness and certitude or certainty of an observable implication that is derived from a hypothesis. Observable implication can refer to anything that can be derived as a prediction from a theory. In the context of process tracing, an observable implication often refers to what is now called a causal process observation (CPO, see Collier, Brady, and Seawright 2004). 3 However, CPOs are only observations and do not stand for themselves. They can be tied to the cause of interest, the outcome, the mechanism linking cause to outcome, or to additional causal relationships that should hold true if the causal relationship of prime interest holds true (Mahoney 2010). Ziblatt s (2009) analysis of electoral fraud in nineteenth-century Germany can serve as an example for the various roles of CPOs. He hypothesizes that electoral fraud in an electoral district is becoming increasingly likely as landholding inequality rises. The so-called capture mechanism that accounts for this link is that landowners capture the local administration in charge of the elections by landowners. The rationale is that landowners feel threated by democratization because it undermines the power and status they derive from their wealth. Ziblatt measures the outcome, electoral fraud, by consulting and coding petitions to the German Reichstag charging electoral fraud in a specific district (Ziblatt 2009: 5-6). 4 In process tracing, the capture mechanism is then accessed via the collection of CPOs such as the observation that a landowner, who was head of the local administration, urged owners of pubs to avoid offering their rooms for meetings of prodemocratization parties (Ziblatt 2009: 16). An example of an auxiliary outcome substantiated with CPOs is that landowners align with other groups that have an equal interest in hindering democratization through the manipulation of elections. Ziblatt delivers CPOs that representatives of the incumbent conservative party, who felt threatened by the transition process and the Social 4

6 Democratic Party, collaborated with the landowners (Ziblatt 2009: 16). 5 CPOs thus can serve measurement purposes and allow us to generate causal inferences on causal effects and causal mechanism. However, following the existing literature on the 2x2 typology, I focus on empirical studies and examples that invoke process tracing in order to collect CPOs substantiating the claim that a cause is connected to the outcome and/or that a causal mechanism is operative. 6 The Ziblatt study can also be used to illustrate the logic behind the 2x2 typology constituted by two dimensions referring to different aspects of an observable implication. The dimension of uniqueness denotes whether a given observable implication is specific to one explanation or can be derived from multiple hypotheses. 7 In Ziblatt s research, both uniqueness and non-uniqueness are given on the level of mechanisms. The capture mechanism entails that a landowner occupies the leading position in the local administration (Landrat) and uses his position to manipulate the elections. This expectation cannot be derived from the social-power mechanism, which is the mechanism against which Ziblatt tests his preferred capture mechanism. According to the social power mechanism, landowners use their social power to threaten and coerce dependents to not vote at all or at least not in favor of pro-democratization parties (Ziblatt 2009: 3). Since the exertion of social power does not require it to become head of the local administration, the two mechanisms yield unique predictions. However, this differs for the conservative-party mechanism as it entails the same observable implication as the capture mechanism. Conservative parties were the parties of the landed elites (Ziblatt 2009: 9). This means the political stakes for the conservative party were particularly high in districts in which the landed elites held a large share of the land, highlighting that the conservative-party mechanism is linked to high levels of landholding inequality as well. Since the heads of the local administration were representatives of the German state, the incumbent Conservative Party faced a strong incentive to manipulate elections by instructing the local administration to do so. Indeed, 5

7 Ziblatt presents evidence that the Conservative Party relied on the local administration for the manipulation of elections on the district level (Ziblatt 2009: 16). The 2x2 typology intersects the dimension of uniqueness with the dimension of certitude or certainty. On this dimension, one asks for the probability with which one expects to find observable implications confirmed. When certitude is high, one is relatively confident of gathering supportive evidence, whereas low certainty means that it is relatively unlikely to collect confirming evidence for a given implication (Bennett 2008: 706; Van Evera 1997: 30). As regards the Ziblatt study, certainty is high for the region that he selects for process tracing because East Elbia is described by a high degree of landholding inequality (Ziblatt 2009: 14). If landowners should have felt threatened by democratization and taken countermeasures, then this should be observable in the region of East Elbia where they had the most to lose. 8 The same holds true for the conservative-party mechanism because the more unequal land distribution, the higher the stakes for the Conservative Party and the more likely it is that the latter manipulates the elections. Correspondingly, landowners are the less likely to subvert the democratization process, the more equally land is distributed in a region. The intersection of the two dimensions produces the four types of hypothesis tests summarized in table 1. The typology is presented in combination with a recent amendment by Bennett (2010: ), asking whether a successful test is a necessary or sufficient criterion for inferring that the hypothesis is correct (see also Collier 2011). This amendment is not to be confused with necessary and sufficient causation in set-relational research because the typology generally applies to all types of hypotheses. This means that the necessary and sufficient criteria for hypothesis testing pertain to hypotheses on necessary and sufficient terms alike. The four types of tests and the criteria of necessity and sufficiency are now clarified step by step. [Table 1 about here] 6

8 Table 1: Types of hypothesis tests and causal inference Uniqueness (passing test is sufficient for inferring causation) High (Yes) Low (No) Certainty (passing test is necessary for inferring causation) High (Yes) Low (No) Doubly decisive test Smoking gun test Hoop test Straw-in-the-wind test 7

9 The weakest of all tests is the straw-in-the-wind test because it is marked by low uniqueness and low certainty (Collier 2011: 826). A test of the capture mechanism against the conservative-party mechanism in a district with low landholding inequality is a straw-in-the-wind test; one is unlikely to find confirming evidence because land is equally distributed, and the implications are not unique because they both predict the same implication. A passed straw-in-the-wind test is not sufficient to infer that the capture hypothesis is correct because the conservative-part hypothesis would be confirmed as well, which is reflected in low uniqueness. Furthermore, a passed test is not necessary for a confirmatory causal inference because failure is of little surprise in light of low certainty. This means that negative evidence of course disconfirms the hypothesis, but that it is still eligible to a follow-up test where the ex ante likelihood of finding it confirmed is high (i.e. we should do a hoop test or a doubly decisive test). A hoop test is characterized by high certainty and no uniqueness. A test of the capture mechanism against the conservative-party mechanism in a district with high landholding inequality represents a hoop test. Certainty is high because one expects to find confirming evidence in such a district, but uniqueness is absent because the two hypotheses entail the same prediction. Similarly to a straw-in-the-wind test, a passed hoop test is not sufficient for inferring causation due to low uniqueness. But passing the test is necessary because failing a test that was likely to be passed casts serious doubt on the hypothesis. Smoking gun tests have high discriminatory power deriving from high uniqueness. It combines with low certainty of finding an implication confirmed. A smoking gun test of the capture mechanism involves the selection of a district with an equal distribution of land and tests this mechanism against the social-power mechanism. A successful smoking gun test is sufficient for a confirmatory causal inference because it lends credence to the capture hypothesis only. At the same time, passing a test is not necessary because failure does not come as a surprise. 9 8

10 Doubly decisive tests are deemed to be the most powerful type because they combine high uniqueness and high certainty (Van Evera 1997: 31-32). With regard to the Ziblatt example, a double decisive test is performed when choosing a district with high landholding inequality for a test of the capture mechanism against the social-power mechanism. The intersection of high uniqueness and certainty means that passing a test is necessary and sufficient for a confirmatory causal inference. According to the conventional reading, supportive evidence for a unique implication with high certainty permits it to make a confirmatory inference for the capture hypothesis, while at the same time refuting the social-power hypothesis (Bennett 2010: 211). The discussion of the four types of tests demonstrates that the realization of one test or the other has very different implications for causal inference. In light of this, one can wonder whether we can navigate between the types by crafting the analysis accordingly. Although this is a salient issue, the literature on the typology hasn t dealt with this matter in a comprehensive manner thus far. The remainder of this paper is dedicated to a discussion of how case selection can be used to adjudicate between the tests. The discussion starts with the dimension of certainty and then turns to the role of uniqueness for causal inference. 3. Certainty and case selection The dimension of certainty is closely tied to matters of case selection. 10 The reason is that a specific case allows us to determine on the basis of theoretical considerations whether an implication entails a high or low certainty of being found confirmed in process tracing. Implicitly, this became apparent in the empirical example when I argued that the analysis of a district with low landholding inequality entails low certitude for the capture hypothesis, while the choice of a district with an unequal distribution of land implies high certainty. When we theorize that the level of landholding inequality has an influence on electoral fraud, the former thus is an 9

11 essential case-specific feature that shapes the likelihood with which we expect to find the hypotheses on the mechanisms confirmed. It follows that the intentional choice of cases with a high or low level of certitude allows it to navigate between a doubly decisive test and a hoop test on the one hand, and a smoking gun test and a straw-in-the-wind test on the other. The case study literature offers two established types of cases that fit squarely into the distinction between high and low certainty (Eckstein 1975). A most-likely case implies a high certainty of gathering particular pieces of evidence, which is equivalent to high certitude. Correspondingly, a least-likely case entails a low likelihood of finding specific evidence (Bennett 2008: 719; Levy 2008: 12-13), translating itself into low certitude. 11 Linkage of the dimension of certainty with the most-likely and least-likely case study demonstrates that the certainty with which one expects to gather observations is neither high nor low per se, but only for a particular case. Concerning the above-mentioned benefits of high certainty, which is equivalent to a necessary element for causal inference, there are apparent advantages to the choice of most-likely cases. However, high certainty might coincide with low uniqueness and leave one with a hoop test only, which allows for less powerful causal inference than a doubly decisive test. Consequently, there are also apparent benefits to making an intentional choice between tests that are characterized by high and low uniqueness, respectively. 4. Uniqueness and contrasts in comparative tests In contrast to the dimension of certainty, the dimension of uniqueness is not automatically tied to matters of case selection. It is possible to specify observable implications and assess their uniqueness without considering any case against which these predictions can be tested. The implication of the capture hypothesis that landowners seize the local administration is unrelated 10

12 to any specific electoral district one could choose for process tracing. However, cases of course must be chosen in order to be able to test the observable implications of a theory. As is demonstrated in the following, case selection is not an easy matter because there is no guarantee that an implication unique to one theory also has high uniqueness for the selected case. With regard to the Ziblatt study, an example of this discrepancy would be the choice of an electoral district in which landowners hold social power and capture the local administration by filling the position of a Landrat. While the social power hypothesis and the capture hypothesis entail a unique implication concerning the mechanism underlying electoral fraud, we have chosen a case that meets the preconditions for a simultaneous test of both hypotheses. This means that in regard to the chosen electoral district, the implications are not unique in the sense that process tracing might deliver empirical evidence that simultaneously substantiates both propositions. Of course, it might be that one finds confirming evidence only for the capture hypothesis or the social-power hypothesis, but this would be due to chance and not due to the purposeful choice of a suitable case. The extant literature on the 2x2 typology simply refers to whether the uniqueness of an observable implication is given or not (e.g. Bennett 2008). However, the example just given demonstrates that the mere reference to the (non-)uniqueness of an implication is ambiguous. The ambiguity can be removed by distinguishing between the theoretical uniqueness and empirical uniqueness of observable implications. Theoretical uniqueness refers to the question of whether one or more hypotheses yield a specific observable implication in principle. Empirical uniqueness asks whether we can test the observable implication of one or more than one hypothesis given a specific case. 12 As indicated above and as I show in the following, theoretical and empirical (non-)uniqueness do not necessarily go together, rendering it important to 11

13 understand their interplay and the role of case selection as the connection between the two forms of uniqueness. 13 In the following elaboration of the interplay between theoretical uniqueness, case selection, and empirical uniqueness, comparative hypothesis testing involves a working hypothesis (W) and a rival (or competing) hypothesis (R). The ten designs presented in table 2 capture all relevant constellations between the working hypothesis and the rival hypothesis by asking whether the two hypotheses center on the same or a different cause or condition, 14 the same or a different mechanism, and same or a different outcome. 15 When the cause, mechanism, or outcome differs between the propositions, the systematization follows the idea of contrast classes. The idea behind contrast classes is to ask why one specific cause or mechanism, rather than another cause or mechanism, is expected to produce a specific outcome, rather than an alternative outcome (see Northcott 2008). The contrast thus lies in the comparison of two states of the cause, mechanism, or outcome, one of which occurred and one of which did not occur in a given case. 16 For contrastive mechanisms, causes, and outcomes, I further distinguish between exclusive and non-exclusive contrasts. In the second column of table 2, I present a formalized view on the constellations between the working and the rival hypothesis. A and B denote conditions or contrast classes that are different, but not exclusive while ~A ( not-a ) signifies a mutually exclusive condition compared to A. Correspondingly, M and N capture mechanisms and ~M denotes a directly contradictory mechanism compared to M. Y and Z mark two potential outcomes and ~Y the mutually exclusive outcome in relation with Y. (An extended formalized account is provided in the online appendix to this paper). 17 The following discussion of the designs and their implications for case selection and causal inference is organized along the dimension of the condition (see below). Following this 12

14 logic, the first four constellations involve working and rival hypotheses centering on the same condition. Among the four designs, scenarios 1a and 1b specifically focus on exclusive and nonexclusive mechanisms. Designs 4 to 6 involve propositions with mutually exclusive conditions (A vs. ~A), while the last three design (7 to 9) include non-exclusive conditions (A vs. B). In order to keep the discussion of the designs comprehensible, only designs 1a and 1b explicitly deal with contrastive mechanisms. Besides that a three-dimensional visualization becomes demanding and potentially more confusing than insightful, 18 table 2 suffices to give a full exposition of the range of constellations one may confront. All the arguments that I make for designs 1a and 1b extend to the other eight designs as they can involve exclusive and nonexclusive mechanism as well. From an inferential point of view, though, contrastive mechanisms are not needed here and thus are not taken into the picture when elaborating designs 2 to 9. While the causal statements are located at the cross-case level for these designs, the case studies that I present in the following are about process tracing on the empirical level because the authors use process tracing to substantiate the claim that there is a causal link between the cause and the outcome. [Table 2 about here] In the following sections, each of the ten designs is illustrated with an empirical study with regard to the following features. First, do the two hypotheses yield unique implications on a theoretical level (column three of table 2)? While, for reasons elaborated above, the certainty for each observable implication matters for causal inference, certainty is not further considered to maintain the full focus on the dimension of uniqueness (see also below). Second, do the two hypotheses imply mutually exclusive, i.e., directly contradictory theoretical implications (column 13

15 four)? Third, I explain what the optimal case selection strategy is, which requires elaboration of what optimal means. 19 The primary goal is to generate an unambiguous causal inference related to the working hypothesis. 20 The focus of this column rests on the working hypothesis because it is, by definition, the hypothesis that one favors and seeks to test. On the one hand, it is evident that empirical uniqueness always allows it produce an inference for the working proposition only, whereas the following sections show that it is sometimes simply impossible to achieve empirical uniqueness via case selection. 14

16 Table 2: Designs for tests of working hypothesis and rival hypothesis Cause Design Outcome Formalization Theoretical uniqueness Mutually exclusive implications Optimal case selection rule Empirical uniqueness for working hypothesis Implications of confirmed working hyp. for rival Same Same (1a) Exclusive mechanisms (1b) Non-exclusive mechanisms W: A M Y R: A ~M Y W: A M Y R: A N Y Yes (mechanisms) Yes (mechanisms) Yes No Case with A/Y present Case with A/Y present and N absent No Yes Disconfirmed None Exclusive (2) Exclusive (3) Non-exclusive (4) Same (5) Exclusive (6) Non-exclusive W: A Y R: A ~Y W: A Y R: A Z W: A Y R: ~A Y W: A Y R: ~A ~Y W: A Y R: ~A Z Yes Yes Case with No Yes Yes Disconfirmed A present No Disconfirmed Yes No No None Yes No Yes Confirmed Yes No Yes None (7) Same W: A Y R: B Y No No Case with A present and B absent Yes None Nonexclusive (8) Exclusive W: A Y R: B ~Y Yes No Case with A and B present No Disconfirmed (9) Non-exclusive W: A Y R: B Z Yes No Case with A and B present No None 15

17 On the other hand, the following sections show that when the achievement of empirical uniqueness is possible, it is not always necessary to establish because it might deny generating inferences on the working hypothesis and the rival account. If we extend the view beyond the working hypothesis and also want to seize the opportunity for inferences on the rival proposition, it follows that empirical uniqueness for the working hypothesis is not necessarily the best strategy to follow (at least as long as this does not conflict with the goal of unambiguous inferences on the working hypothesis). In the following section, I show that for some designs an unambiguous inference on the working proposition requires choosing a case that does not allow us to test the rival hypothesis at the same time. In other designs, however, the intentional choice of a suitable case permits us to test the working hypothesis and the rival proposition simultaneously, which is superior to testing the former hypothesis only. Since it is not always possible or necessary to achieve empirical uniqueness for the working hypothesis via case selection, column six includes information on whether the recommended case selection rule establishes empirical uniqueness for the working hypothesis. With regard to column five, it should further be noted that it specifically yields information on the positive cases that we need for a test of the working hypothesis and, depending on the design, the rival proposition. 21 Here, my understanding of positive case is that of a case displaying the condition or mechanism entailed by the working hypothesis. Epistemologically, of course, causal inference that follows the criterion of difference-making demands a negative case (Northcott 2008), allowing one to assess whether the condition or mechanism makes a difference to the outcome (see Machamer 2004). 22 The negative case can be an actual one (Mackie 1974: chap. 3) or introduced by means of a counterfactual (Lewis 1973), which is a matter that does not need concern us here. (See the online appendix for a discussion of negative cases in all ten designs.) 16

18 Fifth and finally, I clarify the consequences of the confirmation and disconfirmation of the working hypothesis for the rival account, given the theoretical constellation between them and the assumption that the optimal case selection strategy is followed. It generally holds that if a confirmed working hypothesis has no ramifications for the competing account (which holds true for the majority of designs), so does a failed working proposition not entail anything for the rival hypothesis. On the other hand, when the confirmation of a working hypothesis automatically implies the disconfirmation of its competitor, the latter is also directly found confirmed when one gathers disconfirming evidence for the working proposition. I also note that it is possible to elaborate more complex hypotheses and explanations than is done in the empirical examples that are presented below. With respect to table 2, A, M, etc. can generally denote a single mechanism or cause, but could also include conjunctions of conditions or mechanisms, SUIN and INUS causes, explanations involving a fine-grained chain of intervening steps, sequencing, etc. (Mahoney 2000; 2010; Mahoney, Kimball, and Koivu 2009; Pierson 2004). However, the complexity of hypotheses and explanations on effects and mechanism does not change anything about the arguments that I develop in the following. The formulation of a complex explanation means to derive an observable implication, much as the specification of a simple hypothesis stipulating a mechanism. The elaboration of a complex explanation promises inferential benefits because the more specific an explanation is, the more likely it is that it is unique with apparent advantages for causal inference. Still, a salient question is whether the explanation is incompatible (exclusive) or compatible (non-exclusive) with a rival explanation, which is captured by table 2. It is also evident that table 2 only captures the most basic constellations involving two hypotheses positing different conditions, mechanisms, and outcomes. In practice, one is likely to operate with more hypotheses related to each other in multiple, probably complex ways in terms of similar, exclusive, and non-exclusive implications. 17

19 In principle, however, what is elaborated for two hypotheses extends to tests of multiple hypotheses. 23 I further note that if the following discussion explicitly touches on the doubly decisive test as it was introduced above, the assumption is that uniqueness is complemented with high certainty as this combination is constitutive for this test. Furthermore, when the discussion deals with the empirical confirmation of the working and the rival hypothesis, it is also clear that the two might differ in regard to the respective certainty with ramifications for causal inference. 24 In order to highlight the dimension of uniqueness, the implicit assumption is that the two propositions do not display noteworthy differences in the certainty of the respective observable implications. Finally, it should be emphasized that all examples introduced so far and to be introduced below are statements about the sufficiency of a condition. Hypotheses on necessary conditions and correlational cause-effect relationships are not covered because this would require the discussion of 30 empirical examples in this paper. On the practical side, the focus on sufficiency can be justified to the extent that process tracing and case studies often involve statements about sufficient conditions (Blatter and Haverland 2012; Goertz and Mahoney 2012; Mahoney 2004; Mahoney and Goertz 2006). Equally important, the major conclusions that I derive from table 2 fully extend to correlational and necessary-condition case studies, which is elaborated in detail in the online appendix to this paper Working and rival hypothesis with same condition The first two designs in table 2 capture a constellation in which two hypotheses share the same cause and outcome, but differ as to the underlying causal mechanism that can be exclusive (1a) and non-exclusive (1b). An example of design 1a with exclusive mechanisms is Grieco s (1990) 18

20 study of the implementation of the so-called Codes negotiated at the GATT Tokyo Round in the 1970s. 26 In his case study, Grieco is testing neo-liberalism against neo-realism. This test includes contrastive mechanisms because neo-realism claims that states seek relative gains in international cooperation. In contrast, neo-liberalism stipulates that states are satisfied when they can achieve absolute gains. Absolute-gain seekers thus also cooperate with other countries when the latter can reap more benefits from the agreement. These two predictions on the mechanisms of cooperation imply that neo-realism and neo-liberalism achieve theoretical uniqueness and stipulate mutually exclusive implications. 27 Given the presence of contradictor predictions, it is apparent that the confirmation of one hypothesis automatically disconfirms the other. 28 Design 1a thus is in line with the conventional reading of the doubly decisive test. The choice of a suitable case for a comparative analysis is uncontroversial because one only needs to choose one case with the condition and the outcome present; in this example, these are the Tokyo mandate to negotiate about the implementation of the Codes and the actual outcome of the negotiations. Process tracing then allows it to discern whether countries sought absolute or relative gains and to confirm one hypothesis while disconfirming the other. The case that we need to test the working hypothesis and its rival necessarily fails to achieve empirical uniqueness because both propositions predict different mechanisms when the condition and the outcome are present. The situation is different when the two mechanisms are not exclusive, captured by design 1b. Ziblatt s study (2009), introduced previously, is a prime example of a design with nonexclusive mechanisms. As explained, three possible mechanisms, the social-power mechanism, the capture mechanism, and the conservative-party mechanism, link landholding inequality to electoral fraud and exhibit theoretical uniqueness. The salient difference between Grieco s and Ziblatt s case study is that all three mechanisms could find empirical confirmation in Ziblatt s 19

21 analysis. Empirically, it may be that landowners use their social power to manipulate elections; that they additionally take control over the local administration in order to have additional leverage for electoral fraud; and that the incumbent conservative parties have their own interest in manipulating elections via the head of the local administration. Design 1b highlights that theoretically unique implications do not automatically entail contradictions; each hypothesis yields a unique mechanism with unique observable implications, but the confirmation of one hypothesis does not simultaneously disconfirm the other two. The Ziblatt example is an instance of equifinality or substitutability on the level of mechanisms, i.e., multiple mechanisms potentially connect the same cause to the same outcome. In fact, the argument that the confirmation of one hypothesis automatically rejects the other two is a misreading of the hypotheses because each of them allows the other two hypotheses to be empirically accurate as well. As regards the necessary and sufficient criteria for hypothesis testing introduced above, it follows that the confirmation of a hypothesis including a nonexclusive and unique mechanism is not sufficient for confirming this proposition and refuting its rival. The correct interpretation of uniqueness and non-exclusive mechanisms has further implications for the collection of empirical observations and their use for causal inference. In the misleading interpretation of the doubly decisive test, a CPO is taken as supportive evidence for one hypothesis and disconfirming evidence for the other proposition. However, if the implications of two hypotheses are unique, but non-exclusive, it is mandatory to treat each hypothesis on its own empirical ground. One has to search for evidence lending support to each of the two hypotheses and determine their validity separately. The working and rival hypothesis are confirmed by the respective presence of supportive evidence, and disconfirmed by the absence of such evidence or the presence of countervailing evidence

22 Ziblatt is well aware of the relation between his hypotheses and proper collection and interpretation of evidence because he evaluates the social-power hypothesis and the capture hypothesis on separate empirical ground. 30 This means that he is looking for observable implications confirming or disconfirming the capture hypothesis, and is searching for evidence on implications specifically corroborating or disconfirming the social-power hypothesis. In the face of unique and non-exclusive mechanisms, the challenge in creating the best possible basis for causal inference on the working hypothesis rests in achieving empirical uniqueness via case selection. In the context of design 1b, empirical uniqueness entails the choice of a case that only allows us to test for the presence of the mechanism stipulated by the working hypothesis. In Ziblatt s study, empirical uniqueness is absent because the selected region of East Elbia permits it to simultaneously test all three mechanisms. Indeed, Ziblatt finds disconfirming evidence for the social-power mechanism, but is left with supportive evidence for the capture mechanism and the conservative-party mechanism. This study thus highlights that case selection is the transmission belt between theoretical and empirical uniqueness. In Ziblatt s analysis, an unambiguous causal inference would have been feasible by looking for a region where one would expect to find positive evidence for the capture mechanism (his preferred one), whereas the antecedents for a test of the other two mechanisms were not met. Such a region would permit a focus on one hypothesis and the generation of inferences for this hypothesis only. 31 It is true that this analysis would no longer be a comparative test because the social-power hypothesis and the conservative-power hypothesis have been removed from the equation by means of case selection. For design 1b (and other designs that follow), this is the price one has to pay to be certain that high theoretical uniqueness is complemented with high empirical uniqueness in a single-case study. If one wants to perform a real comparative test of the three 21

23 mechanisms, one must select three positive cases, each of which is appropriate for the test of one hypothesis only. These problems of comparative hypothesis testing and case selection are not new to case study research as this is a long-standing topic in the case study literature (Lijphart 1971; Zelditch 1971; Lieberson 1991). However, the role of case selection is worth pointing out because it demonstrates that theoretical uniqueness does not necessarily imply unambiguous confirmatory (or disconfirmatory) causal inferences. When the working and rival hypotheses only distinguish themselves through a non-exclusive mechanism, theoretical uniqueness must be translated into empirical uniqueness by informed case selection. Whether or not the appropriate cases are available is an empirical question, meaning that it or might not be possible to select a case that achieves empirical uniqueness. In principle, though, a reflected choice of cases is the vehicle for avoiding a lack of empirical uniqueness and the corresponding problems for causal inference. Design 2 in table 2 combines two hypotheses that focus on the same cause and include a mutually exclusive outcome. Zuber s (2011) case study on party strategies in ethnically divided societies offers an example of such a design. The starting point of Zuber s analysis is the established hypothesis concerning which competition between political parties representing ethnic minorities leads to the ideological radicalization of these parties. Drawing on the literature on party competition, Zuber conjectures that ethnic minority parties also face incentives for ideological moderation, i.e., taking a position that is closer to the center of the ideological spectrum rather than at the extreme poles. A case study of ethnic-minority party strategies in Serbia shows that ethnic minority parties do opt for ideological moderation and, additionally, sheds light on why this is done. The collection of supportive evidence for ideological moderation directly contradicts the radicalization argument because a party can either moderate or radicalize its position at a given election. 22

24 In this design, theoretical uniqueness is given because both causal arguments center on a different outcome. Moreover, the two propositions are mutually exclusive because only one of the two hypotheses can be found confirmed in a single case. Case selection is straightforward because all one needs to ascertain is the presence of the condition, i.e. that the selected party represents an ethnic minority competing with another minority party. Interestingly, then, the choice of such a case thus necessarily lacks empirical uniqueness because the presence of the condition entails the test of two competing hypotheses. In total, a design with the same cause and mutually exclusive outcomes exhibits the features of a doubly decisive test. Case selection and causal inference follow different lines when the cause is identical and the outcomes are not exclusive (design 3). For an illustration of this design, I rely on Kammer s (2013) research on the consequences of globalization on redistribution in advanced democracies (see also Kammer and Niehues 2013). Kammer develops the hypothesis that globalization has two consequences for redistribution: it leads to less redistribution via the tax system and more redistribution via the welfare system. There is no inherent link between the two trajectories of redistribution, meaning that more or less redistribution (or no change at all) through taxation can coincide with more or less redistribution through welfare spending. 32 Theoretical uniqueness is given for each hypothesis because they contrast different and non-exclusive outcomes, but these propositions do not contradict each other. Again, this shows that theoretical uniqueness is not sufficient for mutually exclusive implications. Theoretical uniqueness can be complemented with a lack of empirical uniqueness because the analysis of a globalized country, i.e. a case with the condition in place, could deliver supportive evidence for both propositions. If one wants to avoid that both hypotheses receive empirical support at the same time, one has to choose a case that is suitable for a test of the working proposition only. For example, one could select a country about which one knows that redistribution through the tax 23

25 system is low at the outset, allowing one to focus on the consequences of globalization on welfare system-based redistribution. 33 But since we are interested in comparative hypothesis testing and are dealing with two non-exclusive outcomes here, the question is whether empirical uniqueness is desirable at all. In other words, we have a discrepancy between case selection achieving empirical uniqueness on the one hand, and the choice of a case permitting us to test both hypotheses. If we choose a case allowing us to test both propositions at the same time, we contribute more to the generation of knowledge than in a design establishing empirical uniqueness for the working hypothesis. In this view, design 3 is one of the constellations for which there are benefits in deviating from the idea of empirical uniqueness Working and rival hypothesis with exclusive X For the exposition of designs 4 to 6, I rely on studies that play exclusive causes against each other. An example mirroring design 4, which involves two propositions with exclusive causes and the same outcome, is Hendriks and Michels (2011) study of the spread of elements of direct democracy in the United Kingdom and the Netherlands. They hypothesize that the advent of direct democracy is attributable to an international debate about direct democracy that equally influences the United Kingdom and the Netherlands and thus renders their political regimes more similar. The comparison of a majoritarian democracy (United Kingdom) with a consensus democracy (the Netherlands) further permits the assessment of the counterclaim that either consensus or majoritarian political regimes are the actual driving force behind the invention of features of direct democracy. 35 For purposes of illustration, I focus on this claim here and take as the working hypothesis that consensus democracies exhibit more elements of direct democracy than majoritarian countries

26 Theoretical uniqueness is not given in this design because both hypotheses entail the same outcome. However, a lack of theoretical uniqueness is not a serious problem for either case selection or causal inference. Design 4 is an instance of a doubly decisive test by choosing a case with the condition of the working hypothesis in place; if we find that consensus democracies have more elements of direct democracy installed than do majoritarian countries, it directly follows that the hypothesis is wrong according to which the majoritarian nature of a political regime is the cause of more direct democracy in a country (and vice versa). 37 With regard to necessary and sufficient criteria for causal inference, design 4 shows that theoretical uniqueness is not necessary for unambiguous causal inference. A successful test of a hypothesis lacking theoretical uniqueness can result in confirmatory causal inferences if it involves two propositions that contrast exclusive causes and center on the same outcome. The discussion of design 5 draws on Eckert s (2010) inquiry of regulatory autonomy in the European Union. Eckert argues that the type of capitalist system in a country influences various dimensions of the autonomy of regulatory agencies in the postal sector. Among other things, Eckert (2010: 1235) hypothesizes that liberal market economies grant encompassing competencies, coordinated market economics confer medium competencies, and that state-led market economies grant limited competencies. Each of the three propositions displays theoretical uniqueness because they settle on different outcomes. In addition, empirical uniqueness is automatically guaranteed for a selected positive case because a country is subsumed under only one variety of capitalism (which is the condition in this example). One case allows it to focus on the corresponding hypothesis on the consequences of this type for regulatory autonomy. With regard to causal inference, the nonexclusivity of the causes and the outcomes implies that all propositions can be empirically valid. In fact, design 5 stands out from the other nine constellations because the confirmation of the 25