Theories of categorical reasoning and extended syllogisms

Transcription

1 THINKING & REASONING, 2006, 12 (4), Theories of categorical reasoning and extended syllogisms David E. Copeland University of Southern Mississippi, Hattiesburg, MS, USA The aim of this study was to examine the predictions of three theories of human logical reasoning, (a) mental model theory, (b) formal rules theory (e.g., PSYCOP), and (c) the probability heuristics model, regarding the inferences people make for extended categorical syllogisms. Most research with extended syllogisms has been restricted to the quantifier All and to an asymmetrical presentation. This study used three-premise syllogisms with the additional quantifiers that are used for traditional categorical syllogisms as well as additional syllogistic figures. The predictions of the theories were examined using overall accuracy as well as a multinomial tree modelling technique. The results demonstrated that all three theories were able to predict response selections at high levels. However, the modelling analyses showed that the probability heuristics model did the best in both Experiments 1 and 2. The goal of this study was to examine inference making in an extended categorical syllogism task. Extended categorical syllogisms are logical arguments that relate entities, or groups, and consist of more than two premises, or categorical statements. Here is an example: All W are X (premise 1) All X are Y (premise 2) No Y are Z (premise 3) Therefore, no W are Z (conclusion) Correspondence should be addressed to David E. Copeland, Department of Psychology, Box 5025, University of Southern Mississippi, Hattiesburg, MS USA. David.Copeland@usm.edu I would like to thank Seth Allen for his assistance in collecting the data. I also wish to thank Steve Boker, Darcia Narvaez, Mike Oaksford, Michael Wenger, two anonymous reviewers, and especially G. A. Radvansky, for their insightful comments throughout this research project. Portions of this research were supported in part by a grant from the Army Research Institute, ARMY-DASW01-99-K Ó 2006 Psychology Press, an imprint of the Taylor & Francis Group, an informa business DOI: /

2 380 COPELAND There are many explanations for how people who are not experienced with logic make categorical inferences. These explanations include the use of rules, both superficial heuristics (e.g., Woodworth & Sells, 1935) and formal logic rules (e.g., Rips, 1994), probabilistic thinking (e.g., Chater & Oaksford, 1999), and representations such as mental models (e.g., Johnson-Laird & Bara, 1984). Research has supported some of these theories more than others, but no definitive answer has yet emerged for how people reason categorically. The current study examined these various theories of categorical reasoning in an extended categorical syllogism task. This is a highly constrained type of inference making, and thus easy to study. Most studies that have examined extended categorical syllogisms (e.g., Frase, 1969) have considered them in the context of a paragraph, rather than in syllogism format (however, see Griggs 1976a). Moreover, previous studies have been limited to the use of a single quantifier, All. This paper considers extended categorical syllogisms in syllogistic format along with some additional elements neglected by past research. This includes additional quantifiers and syllogistic figure differences. CATEGORICAL LOGIC Categorical logic is a type of deductive logic and is typically expressed in the form of a syllogism. In a categorical syllogism, a specific conclusion is inferred from general statements. For example, if it is known that all men are mortal and that Socrates is a man, then it can be inferred that Socrates is mortal. Traditional categorical syllogisms contain two premises, or statements, that describe a relation among three entities. Two entities (e.g., X and Z) are mentioned in different premises and are linked by another entity (e.g., Y) that is mentioned in both of the premises. There are four orders, or syllogistic figures, in which these terms can be expressed in the two premises: (1) XY-YZ (2) YX-ZY (3) XY-ZY (4) YX-YZ Each premise uses a quantifier to describe the relation among the entities. The quantifiers are listed below along with their traditional abbreviations: (1) All (A) (2) No (E) (3) Some (I) (4) Some... not (O) The combination of the syllogistic figures with the four quantifiers leads to a total of 64 traditional categorical syllogisms. Of this total, only 27 have valid conclusions.

3 EXTENDED SYLLOGISMS 381 Extended syllogisms (also referred to as sorites or set inclusions; see Evans, Newstead, & Byrne, 1993) differ from traditional categorical syllogisms in that they have at least three premises that describe the relations between at least four entities. Because of this, there is no upper limit on the number of extended syllogisms with valid conclusions. It should be noted that if none of the premises has the quantifier, All, there cannot be a valid conclusion. However, just because the quantifier All is present does not mean that there is a valid conclusion. For the current study, extended syllogisms were limited to three premises. Also, syllogistic figure was only manipulated for the first two premises. Using these restrictions creates a well-defined and manageable set for studying extended syllogisms. EXTENDED CATEGORICAL SYLLOGISMS Most studies (e.g., Frase, 1969) that have examined extended categorical syllogisms have only used a single extended syllogism: All V are W! All W are X! All X are Y! All Y are Z Studies investigating extended syllogisms have been restricted in that they have only used the quantifier All. However, they have used a number of different tasks to test performance, including having people recall information, verify the truth of statements, or verify whether statements were old or new, presenting the information in the format of a syllogism, and presenting the information as part of a text. There have been mixed results as to how good people are at accurately drawing inferences for extended syllogisms. Studies that have used a recall test (Favrel & Barrouillet, 2000; Frase, 1969, 1970; Griggs, 1977) have shown that people were more likely to report relationships that were explicitly mentioned rather than relationships that required inferences. Also, studies have shown that people are less likely to accurately identify inferences when more premises are involved (Barrouillet, 1996; Favrel & Barrouillet, 2000). Other studies, however, have supported the idea that people can accurately draw inferences from extended syllogisms. First, a study by Griggs (1976b) asked whether statements were old or new. He found that people rated inferences as old at a similar rate as statements that were in the text. Second, when people have a goal of making inferences, they spend more time reading (Carlson, Lundy, & Yaure, 1992). Finally, the format in which extended syllogisms are presented affects performance. Performance improves dramatically when extended syllogisms are presented in the form of a syllogism (Griggs & Warner, 1982) or when they

4 382 COPELAND are re-phrased using the word in (e.g., All X are in Y) (Mynatt & Smith, 1979). These studies suggest that people are capable of drawing inferences, particularly when using a presentation format such as a syllogism. However, these studies have not considered extended syllogisms in the context of current theories of categorical reasoning. Thus, a closer examination of extended syllogisms is necessary. The next section examines three theories of categorical reasoning that have been used to explain performance in different reasoning tasks. THEORIES OF HUMAN SYLLOGISTIC REASONING There are many different theories of how people solve categorical reasoning problems. The following section examines the most prominent of these ideas: mental model theory, formal rules theory, and the probability heuristics model. For the most part, these theories have not been applied to the domain of extended categorical syllogisms (although see Favrel & Barrouillet, 2000, for a consideration of mental models). This section reviews the explanations of traditional categorical reasoning proposed by these theories. There is also a discussion of how these theories can be used to explain performance on an extended categorical syllogism task. Examples of how each theory applies to extended syllogisms are discussed in a later section. Mental model theory The basic idea of mental model theory (Johnson-Laird, 1999) is that people construct mental representations of the situation described, of the state-ofaffairs in the world (Johnson-Laird, 1983; Johnson-Laird & Bara, 1984; Johnson-Laird & Byrne, 1991). These mental representations simulate the possible ways that the terms can relate to one another. These mental models that are constructed can be compared to one another to contrast different situations that might be possible based on the premises. According to mental model theory, people go through a series of stages. First, a model is constructed for the first premise. Next, people add the information from the second premise. In the third stage, people make an inference from their integrated model. This conclusion expresses a relationship that is not explicitly stated in the premises. The final stage consists of testing the conclusion. People search for counter-examples by constructing alternative models (if possible). These alternative models represent different situations from the original. Once these alternative models are constructed, people can determine whether their conclusion is true for all of the models. If not, people can draw a new conclusion and test it with their models.

5 EXTENDED SYLLOGISMS 383 Finally, if there is no conclusion that is true for all of their models, they respond that there is no valid conclusion. According to mental model theory, syllogisms vary as to whether one, two, or three models can be constructed. The more models that can be constructed, the more difficult the syllogism. One-model syllogisms are the easiest because most people should be able to construct a single representation and draw a conclusion from it. Multiple-model syllogisms tend to be more difficult because people might not consider all possible models, there is more opportunity for error when dealing with more models, and because coordinating multiple models is very taxing for working memory (e.g., Copeland & Radvansky, 2004). Syllogistic figure effects The syllogistic figure effect (Johnson-Laird & Bara, 1984) is the finding that people prefer X-Z conclusions for syllogistic figure 1 (XY-YZ), and Z-X conclusions syllogistic figure 2 (YX-ZY). The explanation for this is that for syllogistic figure 1, people can construct a model of the first premise in the order X-Y. Then, the model for the second premise Y-Z can link onto the Y from the first model. When the integrated model is considered, people can interpret it in that order (i.e., X-Y-Z), which leads to an X-Z conclusion. Syllogistic figure 2 is similar to syllogistic figure 1, except that the model of the first premise, Y-X, gets linked to the end of the model for the second premise, Z-Y. For the symmetrical syllogistic figures, 3 (XY- ZY) and 4 (YX-YZ), there is a slight preference for X-Z conclusions. This is based on the idea of a first-in first-out principle (Johnson-Laird & Bara, 1984). To reduce the load on working memory, people interpret terms from the model in the same order in which they were constructed. The conclusion then uses the terms in this order. Mental model theory and extended syllogisms For mental model theory, people begin to solve syllogisms by constructing a mental representation of the first premise. Then, they expand their model by adding information from the second premise. For extended syllogisms it is assumed that people will simply expand their model a second time by adding the information from the third premise to their model. If there are additional premises, the model will continue to expand to include the new information. Formal rules Formal rule theories consider human reasoning to be based on an internal set of formal inference rules (Braine, 1978; Johnson-Laird, 1975; Osherson,

6 384 COPELAND 1976; Rips, 1983). Different versions of formal rule theories have existed over the years, and they consider reasoning to be a process of constructing mental proofs. These proofs manipulate propositions through the use of rules to reach a conclusion. This section focuses exclusively on Rips (1994) psychology of proof model, typically abbreviated as PSYCOP, because it accounts for the use of quantifiers, as in categorical reasoning. This model is designed to test, or verify, given conclusions. To do this, PSYCOP can use rules that work in a forward direction, backward direction, or both. When moving forward, the model uses rules to generate conclusions. Then it can compare these conclusions to the given conclusion. If this is not successful, then the model can work backward. When it moves in a backward direction, the model begins with the given conclusion and attempts to prove the premises. If a predetermined number of steps is reached by the model, or if no new conclusions are reached, then the model gives up, assuming that there is no valid conclusion. However, conclusions are not always provided with a set of premises. Many studies (e.g., Johnson-Laird & Bara, 1984) require people to generate their own conclusion. Rips outlined a procedure that people follow under these circumstances. The first step that people take is that they apply forward rules in an attempt to generate a conclusion. If this is successful, that is the response. However, if it is not successful, people will employ a strategy, referred to as a dominating heuristic, which is very similar to the matching heuristic (Gilhooly, Logie, Wetherick, & Wynn, 1993). The matching heuristic states that people prefer to use the most conservative quantifier from the premises in the conclusion. The quantifiers rank from most to least conservative, as follows: No, Some... not, Some, and All. Once a conclusion is created, people can try to use both forward and backward rules to verify it. If it is verified, it is the response. If it is not verified, then people can either stick with that conclusion, in the hope that there is some way in which it could be proven, or respond that there is no valid conclusion. Formal rules theory and extended syllogisms In terms of formal rules, research using a propositional reasoning task with multiple premises has suggested that people work through the task in stages (Braine et al., 1995; O Brien, Braine, & Yang, 1994). Braine and his colleagues had people write down inferences as they worked through propositional reasoning problems. The results showed that people not only reached a conclusion, but they were also drawing inferences as they progressed through the premises. That is, people would draw an inference from the first two premises, then they would use that inference, along with the third premise, to draw another inference. After all of the premises were used, the final inference was the conclusion. Thus, according to formal rules

7 EXTENDED SYLLOGISMS 385 theory, people only work with two premises, or one inference and a premise, at a time. While this assumption of serial processing is not a direct prediction of Rips (1994) PSYCOP model, it has been made for this study to allow for predictions in an extended syllogism task. Probability heuristics model The probability heuristics model (Chater & Oaksford, 1999) proposes that people do not reason logically, but instead rely on heuristics. According to this model, people progress through several stages. In the first stage, people use heuristics to generate a conclusion. After this is completed, people move to the second stage, where they test their conclusion. When presented with a syllogism, the first step is to quickly generate a conclusion. To accomplish this, people apply the min heuristic. This heuristic ranks the quantifiers from most to least informative: All, Some, No, and Some... not. The theory states that people prefer to use the least informative of the two quantifiers from the premises, because it is more likely to be true. The premise that is less informative is referred to as the min premise, while the other is the max premise. After the min heuristic is applied, people have the option of applying a second heuristic, called p-entailments, for probabilistic entailments. According to this heuristic, people draw a simple inference that is probably true, based on the initial conclusion. For example, if the initial conclusion was Some X are Z, it is probably true that Some X are not Z as well. Next, people use a third heuristic, attachment, to determine the order of the terms. This heuristic states that if the subject term (i.e., first position) in the min premise (i.e., less informative) is used in the conclusion, then the quantifier in the conclusion attaches to it. In other words, that term will also be the subject of the conclusion. If, however, this does not apply, then the other term (i.e., from the other premise) will be the subject of the conclusion. After a conclusion is generated, people can test, or reconsider the conclusion by using two more heuristics. The first is the max heuristic. According to this heuristic, people should vary as to how confident they are in their conclusion according to the informativeness of the premises. Specifically, the more informative the max premise, the more confidence there is in the conclusion. The less confident a person is, the more likely the person will respond that there is no valid conclusion. The second test heuristic, the o-heuristic (i.e., Some... not heuristic; term O from the A, I, E, O labelling), states that people should avoid conclusions with the quantifier Some... not, because that quantifier is so uninformative. Instead of using this quantifier in the conclusion, people will respond that there is no valid conclusion.

8 386 COPELAND Probability heuristics model and extended syllogisms Because all of these heuristics are only applicable to pairs of premises, it is assumed that they are applied in the same manner as with formal rules theory. That is, they are first applied to premise one and two, and then to the intermediate inference (from the first two premises) and the third premise, for a final conclusion. Formal rules theory and the probability heuristics model are similar in that rules and heuristics are applied to a propositional representation. EXTENDED SYLLOGISMS AND THEORIES OF SYLLOGISTIC REASONING To test predictions made by these theories, two experiments were conducted in which people were presented with three-premise extended syllogisms. In Experiment 1 the extended syllogisms were presented in a format similar to the syllogistic figure 1, and in Experiment 2 they were presented in different syllogistic figures. The specific predictions of each theory were tested by determining the percentage of accurately predicted responses. These values were then adjusted to compute weighted scores, which take into account the number of predicted responses made by each theory. The predictions were also tested by constructing parameterised-computational models (Batchelder & Riefer, 1999), an approach similar to that taken by Oaksford and Chater (1998) and Rips (1994). EXPERIMENT 1 People were presented with a set of 32 extended syllogisms. These syllogisms consisted of three premises and used the following quantifiers: All, Some, No, and Some... not. Nine of them had valid conclusions. All of the syllogisms used in Experiment 1 were presented in a format consistent with syllogistic figure 1: WX-XY-YZ. This is the same format as has been used in previous work with extended syllogisms. The purpose of Experiment 1 was to determine how well people solve extended syllogisms that use quantifiers other than All. In addition, Experiment 1 tested the predictions of three theories for these extended syllogisms. This was done by examining the percentage of responses that corresponded to the responses predicted by each theory. Also, multinomial processing tree model representations were constructed to inspect the response predictions more closely, as well as to examine the probability that certain procedures occur while solving a problem. This is an important test for the theories because it calculates the probability of each response for every extended syllogism, which can be compared with the actual percentages of responses.

9 EXTENDED SYLLOGISMS 387 The models were constructed by taking the steps that occur, according to each theory, and making them parameters in the model. Steps that always occur have parameter values equal to 1, and steps that are probabilistic have values between and For example, consider a three-model extended syllogism explained in terms of mental model theory. According to this theory, people always construct the first model (based on all of the premises). Then, they may or may not construct the second model (i.e., an alternative representation of the premises). If a second model was constructed, people may or may not construct the third model (i.e., another alternative representation of the premises). Finally, the person can provide a conclusion with a W-Z or Z-W order. For this example, the probability that someone responds with an W-Z conclusion from the second model is equal to: P(1 st model) * P(2 nd model) * (1 7 P(3 rd model)) * (P(W-Z)). An example, based on a specific extended syllogism, for each of the theories can be found in Appendix A. The next section examines the predictions made by mental model theory, formal rules theory, and the probability heuristics model. Predictions Mental model theory. According to mental model theory, people construct a mental model of a premise and then integrate additional information into that representation. The predictions for mental model theory were derived from the Syllog program (Johnson-Laird, 1992). The Syllog program assumes an initial interpretation of each of the quantifiers, and this forms the basis for the initial models that people construct. Because Syllog only constructs models for pairs of premises, an additional term, from the third premise, had to be added to models that Syllog originally constructed from the first two premises. For example, consider the following syllogism: Some W are X All X are Y No Y are Z According to Syllog, the following model would be constructed for the first two premises: WXY W XY Again, each line represents an occurrence of an individual (e.g., the second line represents a W that is not an X or Y ). When adding the third premise it is clear that every Y is not a Z, but it is unclear where a

10 388 COPELAND Z, or multiple Z s, can be placed. Thus, the following models are possible: WXYØZ W X Y ØZ W X Y ØZ W W Z W Z XYØZ X Y ØZ X Y ØZ Z Z The first model would predict a possible response of No W are Z or No Z are W. If the second model is considered, there could be a W that is a Z, but this is not certain. The only things that are definitively known are that Some W are not Z and Some Z are not W. Finally, the last model supports the conclusion Some W are not Z. There is a difference if considering the model in a backward direction. In this last model, there is no longer a Z that is not a W. This conflicts with the previous models. So, if someone was reading the model backwards, he or she might respond that there is no valid conclusion. In the above example there are three possible models that can be constructed. While it may be possible that people could construct the third model as their initial model, hence changing their initial conclusion, the Syllog program describes a specific ordering for model construction based on a basic interpretation of the quantifiers. Alternative (i.e., second and third) models, according to Syllog, are attempts to falsify the initial model and its resulting conclusion. The predictions for mental model theory in the current study, in an attempt to stay as true as possible to the original theory, are based on the ordering described in the Syllog Program. 1 For the parameterised computational model, some of the parameters, (i.e., the probability of constructing one, two, and three models) are dependent on each other. That is, the probability that someone constructs a third model can never be greater than the probability that someone constructs the second model. In order to build a third model, you must have already constructed the second one. The parameters for constructing models reflect this dependency: (1) The probability of constructing the first model. (2) The probability of constructing the second model, given that the first was constructed. (3) The probability of constructing the third model, given that the second was constructed. 1 The construction of mental models can be interpreted as being much more complex than the simple idea of constructing one versus two versus three models. However, the fleshing out of the specific aspects of model construction is beyond the scope of this paper. There is a parameterised computational version of mental model theory that is currently being developed which considers the processes involved with constructing mental models in detail (see Copeland & Radvansky, 2005).

11 EXTENDED SYLLOGISMS 389 It is important to note that this does not mean that it is always more likely that people only construct one model, as opposed to two models. For example, if the probabilities that someone constructs the first and second models are 1.00 and 0.60, respectively, then the probability that someone constructs two models would be 0.60 [i.e., P(1 st model) * P(2 nd model j 1 st model)] while the probability of only constructing one would be 0.40 [i.e., 1 7 (P(1 st model) * P(2 nd model j 1 st model))]. Thus, the probabilities of constructing one versus two models are not restricted, only the probabilities of constructing the first versus the second model. The same idea is true concerning a third model. Mental model theory predicts that there should be a high probability that people give W-Z conclusions. The reason is that the order of the terms in syllogistic figure 1 are easy to link up (i.e., W-X! X-Y! Y-Z) in the same order that they are presented. Because of this, the parameter for conclusion direction is restricted to values at or above Formal rules. The predictions of formal rules theory are based on the rules that people have available to use. However, as Rips (1994) points out, and as later emphasised by Johnson-Laird (1997), individuals can differ in the rules that they possess. Because the current study is not investigating individual differences, a standard set of predictions is tested. Rips (1994) specifies that for any pair of premises, the responses should consist of (a) conclusions that follow from the application of forward rules, (b) conclusions that use the dominant quantifier from the premises, when guessing or using backward rules, (c) no valid conclusion, and (d) a small number of correct conclusions with nondominant quantifiers, based on backward processing by the persistent subjects (p. 247). In a footnote (p. 409), Rips explains that conclusions that result from the application of forward rules use the dominant quantifier from the premises. Because the dominant quantifier is not always the correct conclusion, and there is a possibility that people are persistent, there are always three, and sometimes four, conclusions predicted for every pair of premises: (1) The dominant quantifier in a forward direction (i.e., X-Z). (2) The dominant quantifier in a backward direction (i.e., Z-X). (3) No valid conclusion. (4) The correct conclusion when it differs from the dominant quantifier. Here, it is assumed that people draw an intermediate, or temporary, conclusion from the first two premises. Then, that conclusion is used as a premise, along with the third real premise to draw a final conclusion. Thus, it is possible for there to be more than four predicted conclusions for

12 390 COPELAND an extended syllogism. This is because three or four temporary conclusions are possible for the first two premises, and then each one of those is matched with the third premise. For PSYCOP, the probability of successfully using forward rules, if applicable, should be high. This set of rules is simple and the parameter value for it was fairly high in a study conducted by Schank and Rips (as reported in Rips, 1994). The probability of successfully using backward rules is expected to be lower. There are more of them and they are more complex than forward rules. Rips (1994) explained that approximately 32% of subjects from Johnson-Laird and Bara s (1984) study produced conclusions that would have been obtained by using backward rules. In the parameterised computational model, the value for using forward rules is set at or higher, which makes it unnecessary to restrict the parameter for backward rules. The probability that someone would be persistent that is, prove a conclusion was true even though that conclusion would not have been initially proposed as a tentative conclusion should be very low, so this parameter is restricted to a value less than In addition to using rules to solve syllogisms, PSYCOP states that people might guess. In the study conducted by Schank and Rips (as reported in Rips, 1994), the probability of guessing was quite low, ranging from 8% for guessing All and No to 31% for guessing Some and Some... not. One reason why these values were so low is that people were warned that most categorical syllogisms have no valid conclusion. This probably reduced the likelihood that people would guess. Because the current study does not warn people about the number of extended syllogisms with no valid conclusion, people might be more likely to guess. Thus, the parameter values for guessing are only restricted in that the values for guessing All and No must be smaller than the values for guessing Some and Some... not. There are two additional parameters that are listed, but do not vary. The first refers to producing a tentative conclusion to test. It is assumed that this always occurs because the experiments in this study do not use a verification task. The second is labelled NVC Reverse. This parameter refers to situations in which a person encounters no valid conclusion when considering the terms in one direction (e.g., A-D). When this occurs, it is assumed that people then try the reverse ordering (e.g., D-A). Probability heuristics model. The predictions for the probability heuristics model were derived by first applying the heuristic rules to the first two premises. This led to four different conclusions, two obtained by using the min heuristic and the others by also applying the p-entailment heuristic (the ordering of the terms in each conclusion depends on whether the attachment heuristic is applied). Then, separately, these four conclusions

13 EXTENDED SYLLOGISMS 391 were used as a premise along with the third premise to draw final conclusions. To illustrate these steps, consider the following extended syllogism: Some W are X All X are Y No Y are Z If the min heuristic were applied to the first two premises, the conclusions would be Some W are Y (attachment heuristic is applied) and Some Y are W (attachment is not applied). If the p-entailment heuristics were applied to the first two premises, the conclusions would be Some W are Y (attachment) and Some W are not Y (no attachment). Before continuing to the third premise, it is possible that people might not be confident with their temporary conclusion if they apply the max heuristic or the o-heuristic. If so, people would decide that there was no valid conclusion and stop. However, if people are confident in their temporary conclusion then they can continue to the step of pairing the temporary conclusion with the third premise, as shown below: Some W are Y Some Y are W Some W are not Y Some Y are not W No Y are Z No Y are Z No Y are Z No Y are Z Again, the min and p-entailment heuristics can be applied. For the first pair of premises, the conclusions are No W are Z, No Z are W, Some W are not Z, and Some Z are not W. For the second pair, the conclusions are the same as for the first pair. For the third pair, the conclusions are Some W are not Z, Some Z are not W, Some W are Z, and Some Z are W, and for the fourth pair, Some Z are not W, Some W are not Z, Some Z are W, and Some W are Z. Therefore, based on the application of the min and p-entailment heuristics, there are six possible conclusions: Some W are Z, Some Z are W, No W are Z, No Z are W, Some W are not Z, and Some Z are not W. In addition, the response of no valid conclusion is always possible, because it depends on a person s confidence in their response. Thus, the probability heuristics model predicts seven possible responses to this extended syllogism. For the probability heuristics model, the probability of applying the min heuristic is set at 1.000, meaning that it is always applied. The probability heuristics model does not specify any differences in probability for the probabilistic entailments (e.g., All! Some), but it does state that it is the next most preferred conclusion after the min conclusion. This suggests that the parameter values for the probabilistic entailments should be less than The value for the attachment heuristic is allowed to vary.

14 392 COPELAND The max heuristic states that a person should be confident in the generated conclusion in proportion to the informativeness of the most informative premise (i.e., the max premise). Thus, it is expected that people are the most confident for the max quantifiers in the following order: All! NVC, Some! NVC, No! NVC, Some... not! NVC. For the current study, the parameter values were restricted so that the values observed this order (except for a small addition discussed in the next paragraph). Finally, the o-heuristic states that people should avoid producing or accepting Some... not conclusions because they are so uninformative. This means that most of the time, people should respond that there is no valid conclusion rather than using the quantifier Some... not. Because this heuristic is very similar to the one for Some... not! NVC discussed in the previous paragraph, the ordering in the previous paragraph included the addition of value of the o-heuristic to the Some... not! NVC heuristic. Method Participants. A total of 47 undergraduates from the University of Notre Dame participated in exchange for class credit. The only restriction was that participants must not have taken a course in logic. The data from two participants were excluded for having multiple response times less than 1 second (13 trials for the first and 3 trials for the second), leaving a total of 45 participants. Materials and procedure. A set of 32 extended categorical syllogisms was used in Experiment 1 (see Appendix B). These extended syllogisms were selected based on three ideas. First, it was attempted to represent all of the quantifiers. Second, there should be a low percentage of valid extended syllogisms. Finally, all extended syllogisms were based on syllogistic figure 1 because this is what has traditionally been used in studies examining extended syllogisms. The terms used in each syllogism were based on occupations, hobbies, or interests (e.g., joggers, cyclists, coffee drinkers, etc.). These terms were randomly assigned to each syllogism for each participant. An example of a possible syllogism is: All joggers are students All students are coffee drinkers No coffee drinkers are smokers In addition, people were instructed that each extended syllogism was referring to a room full of 100 people who were randomly selected. This method was used to ensure that each of the situations that were described was plausible to the reader.

15 EXTENDED SYLLOGISMS 393 Before the task, it was explained what an extended syllogism was, and people were instructed to select a conclusion that was definitely true, not simply possible. The premises of each syllogism were presented on the computer screen along with nine possible conclusions. Eight of the choices were created by making two choices of each form (i.e., A, E, I, and O); one with the subject term first and one with the reference term first (e.g., All chefs are bikers, and All bikers are chefs). The ninth choice was No valid conclusion. The premises and the choices were presented on the screen together and people had an unlimited amount of time to select their response. The choices were always presented in the same order. Results Overall, people were able to solve extended syllogisms (M ¼ 39%, SD ¼ 17) at a level comparable to performance for traditional syllogisms in other studies (e.g., M ¼ 43%, SD ¼ 10 for Copeland & Radvansky, 2004). In Experiment 1, the numbers of responses that corresponded to each theory were calculated (see Table 1). However, because there are differences between the theories in the number of predicted responses for each extended syllogism, weighted scores were calculated for each of the theories (see Table 2). This was done because a theory is more likely to predict a response accurately if it makes more predictions than another theory. Weighted scores were calculated using the following formula (note: if a prediction is incorrect, that particular syllogism contributes zero to the total weighted score): weighted score ¼ S (1 7 (# of predicted responses for an extended syllogism/9)). For example, if a theory made 5 predictions for a particular extended syllogism and successfully predicted a response for it, then 4/9 (i.e., 1 7 (5/9)) would be added to the weighted score for that theory for that person. The probability heuristics model had a weighted score smaller than both mental model theory, t(44) ¼ 13.31, p 5.001, and PSYCOP, t(44) ¼ 11.35, p 5.001, which did not differ from one another, t 5 1. TABLE 1 The percentage of responses that corresponded to the predictions made by mental model theory, PSYCOP, and the probability heuristics model in Experiments 1 and 2 (standard deviations are in parentheses) Experiment 1 Experiment 2 Overall Overall Syl Fig 1 Syl Fig 2 Syl Fig 3 Syl Fig 4 MMT 80% (13) 88% (10) 91% (12) 87% (14) 87% (14) 88% (13) PSYCOP 75% (13) 88% (10) 87% (13) 87% (12) 87% (14) 89% (14) PHM 97% (3) 98% (4) 98% (5) 98% (5) 97% (9) 97% (6)

16 394 COPELAND TABLE 2 Weighted scores for mental model theory, PSYCOP, and the probability heuristics model in Experiments 1 and 2 (standard deviations are in parentheses) Experiment 1 Experiment 2 Overall Overall Syl Fig 1 Syl Fig 2 Syl Fig 3 Syl Fig 4 MMT 14.4 (2.3) 15.2 (1.8) 4.0 (0.5) 3.7 (0.6) 3.6 (0.6) 3.9 (0.6) PSYCOP 14.3 (2.6) 16.4 (1.8) 4.2 (0.6) 4.0 (0.6) 4.3 (0.7) 3.9 (0.6) PHM 9.9 (0.4) 9.5 (0.5) 2.4 (0.1) 2.4 (0.2) 2.3 (0.3) 2.3 (0.2) Parameterised computational models. The third way in which the predictions were compared was by constructing a parameterised computational model for each theory. Each model was constructed using the proportions of responses from all participants (i.e., models were not created for each person). Parameters were fixed, allowed to vary from to 0.999, or restricted to a more specific range as discussed earlier. For each theory, a parameter only had one value (e.g., for the probability heuristics model, the parameter value for Attachment was the same for each extended syllogism). Comparisons of the parameterised computational models were made using a corrected version (for smaller samples) of Akaike s Information Criterion (AIC C ; Akaike, 1974; Hurvich & Tsai, 1989) as described by Burnham and Anderson (2004): AIC C ¼ 2lnLþ 2v þ½ð2vðv þ 1ÞÞ=ðn v 1ÞŠ where L is the model s best likelihood and v represents the number of estimated parameters. The formula for the model s best likelihood is given below: lnðlþ ¼S f ij ln½pðr j js i ÞŠ where f ij is the observed frequency where the response, R j, was given when a particular stimulus, S i, was presented (as described by Thomas, 2001). The AIC C score is useful because it determines the best fit while also taking into account the number of parameters used in a model. A lower AIC C score indicates a better fit. The parameters for each model were estimated by using the Solver Program in Microsoft Excel. The computed AIC C for the probability heuristics model (AIC C ¼ ) was smaller than for mental model theory (AIC C ¼ ), which was smaller than for PSYCOP (AIC C ¼ ). The probability heuristics model had the best fit even when considering the number of parameters.

17 EXTENDED SYLLOGISMS 395 In addition to the AIC C scores, there were some parameters (see Tables 3, 4, and 5) that were different from those reported in past studies (Rips, 1994). 2 These were the guessing parameters in PSYCOP. In this experiment, the values were much higher than those described by Rips (1994). These higher values may be a result of the fact that people were not warned that most problems had no valid conclusion and so were more likely to guess. Discussion While the probability heuristics model predicted a larger percentage of responses, the weighted scores suggested that mental model theory and PSYCOP did a better job than the probability heuristics model. Specifically, it seemed as if the probability heuristics model was simply benefiting from predicting a large number of responses relative to the other two theories. However, the parameterised computational models showed that the probability heuristics model, while making a large number of predictions, is able to do a good job of accounting for the probability of each predicted response. Thus, based on the modelling results, the probability heuristics model outperformed mental model theory and PSYCOP in Experiment 1. However, it should be noted that only syllogistic figure 1 extended syllogisms were tested in Experiment 1. The aim of Experiment 2 was to go beyond this. EXPERIMENT 2 People were presented with a set of 36 extended syllogisms. These syllogisms consisted of the nine valid syllogisms from Experiment 1 and their corresponding syllogisms in syllogistic figures 2, 3, and 4. Below are examples of an extended syllogism from each: (1) All W are X (2) All X are W (3) All W are X (4) All X are W All X are Y All Y are X All Y are X All X are Y All Y are Z All Y are Z All Y are Z All Y are Z The purpose of this experiment was to examine whether manipulating the order of the terms affected performance. As in Experiment 1, predictions of mental model theory, formal rules theory, and the probability heuristics model were tested. 2 Some people may find it confusing that for mental model theory, the P(3 rd model j 2 nd model) was larger than the P(2 nd model j 1 st model). However, as explained earlier, these are conditional probabilities, not simply the probabilities of constructing a third versus a second model. The actual probability of constructing a third model is equal to P(1 st model) * P(2 nd model j 1 st model) * P(3 rd model j 2 nd model).

18 396 COPELAND For the parameterised computational models, the only change from Experiment 1 is that here mental model theory has an additional parameter. Specifically, mental model theory predicts that people should have a slight preference for W-Z over Z-W conclusions in syllogistic figures 3 and 4, according to the principle of first-in, first-out (Johnson-Laird & Bara, 1984). Method Participants. A total of 54 undergraduates from the University of Notre Dame participated in exchange for class credit. They had not taken a course in logic and did not participate in Experiment 1. The data from one participant were lost due to a computer error, leaving a total of 53 participants. Materials and procedure. The extended syllogisms used in Experiment 2 are listed in Appendix B. The same procedure that was used in Experiment 1 was used here. Results and discussion The data from Experiment 2 were analysed in the same manner as in Experiment 1. Overall, people reached the correct conclusion 35% of the time (SD ¼ 17). They were more accurate for extended syllogisms in syllogistic figure 1 (M ¼ 44%, SD ¼ 17) than 2 (M ¼ 29%, SD ¼ 21), t(52) ¼ 5.30, p 5.001, 3 (M ¼ 33%, SD ¼ 24), t(52) ¼ 3.56, p 5.01, and 4 (M ¼ 34%, SD ¼ 23), t(52) ¼ 3.53, p Performance for syllogistic figure 4 was better than 2, t(52) ¼ 2.18, p 5.05, and there were no differences between 3 and 4, t 5 1, and 2 and 3, t(52) ¼ 1.53, p ¼.13. An important factor that might have influenced these patterns is that the syllogistic figures had 0, 5, 5, and 3 invalid extended syllogisms, respectively. Thus, people were most accurate for those syllogistic figures (i.e., 1 and 4) with the fewest extended syllogisms that required a response of No valid conclusion. This is consistent with studies examining traditional syllogisms (e.g., Copeland & Radvansky, 2004). That is, people have a tendency to respond with No valid conclusion less often than they should. Overall, the weighted score for PSYCOP was larger than for mental model theory, t(52) ¼ 8.45, p 5.001, and both were significantly larger than the probability heuristics model, t(52) ¼ 34.65, p 5.001, and t(52) ¼ 27.73, p 5.001, respectively (see Table 2). For syllogistic figure 1, PSYCOP had a weighted score that was larger than mental model theory, t(52) ¼ 2.70, p 5.01, and both were larger than the probability heuristics model, t(52) ¼ 22.07, p 5.001, and t(52) ¼ 25.30, p For syllogistic figure 2, PSYCOP had a larger weighted score than mental model theory,

19 EXTENDED SYLLOGISMS 397 t(52) ¼ 4.64, p 5.001, which had a larger weighted score than the probability heuristics model, t(52) ¼ 16.80, p For syllogistic figure 3, PSYCOP had a larger weighted score than mental model theory, t(52) ¼ 10.88, p 5.001, which had a larger weighted score than the probability heuristics model, t(52) ¼ 18.08, p Finally, for syllogistic figure 4, both PSYCOP and mental model theory, which did not differ, t(52) ¼ 1.00, p ¼.32, had larger weighted scores than the probability heuristics model, t(52) ¼ 22.63, p 5.001, and t(52) ¼ 22.99, p 5.001, respectively. Both mental model theory and PSYCOP did a good job of predicting responses to extended syllogisms when correcting for the number of predictions made by each theory. These results are somewhat similar to those observed in Experiment 1, where PSYCOP and mental model theory did not differ, but both did better than the probability heuristics model. To further test these theories, parameterised computational models of each theory were examined. Parameterised computational models. Listings of the parameters for each theory are in Tables 3, 4, and 5. As can be seen, these parameter values are similar to those observed in Experiment 1. The AIC C score for the probability heuristics model (AIC C ¼ ) was the smallest, indicating a better fit than mental model theory (AIC C ¼ ) and PSYCOP (AIC C ¼ ). This is the same pattern as in Experiment 1. However, all of these AIC C values were larger than those from Experiment 1, especially for the probability heuristics model. This suggests that all of the theories had more difficulty accounting for performance when more than one syllogistic figure was used. In both Experiments 1 and 2, the probability heuristics model had the best fit. However, at this point it is unclear why it did better. Thus, it is important to examine the theory predictions and the data more closely. The first, and most important, problem that both mental model theory TABLE 3 Parameter values for mental model theory from Experiments 1 and 2 Experiment 1 Experiment 2 Model Construction P(1 st model) P(2 nd model j 1 st model) P(3 rd model j 2 nd model) Model Evaluation P(W-Z order) Syl Fig Syl Figs 2, 3, & 4 n/a 0.598

20 398 COPELAND TABLE 4 Parameter values for PSYCOP from Experiments 1 and 2 Experiment 1 Experiment 2 Production P(Forward Rules) P(Tentative conclusion) P(W-Z order) P(NVC Reverse) Verify/Proof P(Backward Rules) P(Persistent) No Proof P(Guess All) P(Guess Some) P(Guess No) P(Guess Some... not) (Experiment 1: M ¼ 4.2, SD ¼ 1.0; Experiment 2: M ¼ 4.6, SD ¼ 0.8) and PSYCOP (Experiment 1: M ¼ 3.6, SD ¼ 0.8; Experiment 2: M ¼ 4.3, SD ¼ 0.7) have is that they do not make as many predictions for each syllogism as the probability heuristics model (Experiment 1: M ¼ 6.1, SD ¼ 1.0; Experiment 2: M ¼ 6.6, SD ¼ 0.8). In Experiments 1 and 2, there was a mean number of 6.69 (SD ¼ 0.93) and 7.72 (SD ¼ 0.88) different responses given per extended syllogism (there are nine possible responses for each). Thus, for each theory to adequately predict performance, they need to predict a wide variety of responses. Mental model theory and PSYCOP clearly did not do as well in this regard compared to the probability heuristics model. In Experiment 1 there were additional problems with mental model theory and PSYCOP. One problem was the lack of a specific prediction for these theories. A closer examination of Experiment 1 shows that there is a consistent pattern of predictions made only by the probability heuristics model. When the quantifier Some...not was in at least one premise (11 of 32 extended syllogisms), the probability heuristics model consistently predicted a response using the quantifier Some. Mental model theory and PSYCOP never made this prediction. For these 11 extended syllogisms, the predicted Some response was made by people almost a quarter of the time (M ¼ 22.6%, SD ¼ 13.3). Again, for these extended syllogisms, only one of the theories predicted a response that was frequently made. Both mental model theory and PSYCOP have a major flaw in that they fail to make this particular prediction involving Some...not. In defence of mental model theory and PSYCOP, a somewhat simple explanation for these results can be proposed. It is possible, and very likely