Multilevel Latent Class Analysis: an application to repeated transitive reasoning tasks

Transcription

1 Multilevel Latent Class Analysis: an application to repeated transitive reasoning tasks Multilevel Latent Class Analysis: an application to repeated transitive reasoning tasks

2 MLLC Analysis: an application to repeated transitive reasoning tasks 3 Multilevel Latent Class Analysis: an application to repeated transitive reasoning tasks Abstract In the last decade, both multilevel and latent class models have gain popularity among applied psychological researchers. Recently, a combination of these two approaches was proposed in the sociological methodology and statistical literature that offers even more possibilities for modeling rich psychological data structures. In this article we introduce this multilevel latent class model to explain individual differences between children in the performance in transitive reasoning. According to our theoretical model the likelihood of performing memory and transitivity test-pairs correctly depends on the verbatim and gist trace levels used during the task concerned, whereas the likelihood of using particular verbatim and gist trace levels depends on a child s underlying verbatim and gist ability levels. This is a multilevel explanation that requires a multilevel model to be tested. The data collected for our study is a multilevel data set in which tasks are nested within children and test-pairs are nested within tasks. For our data we found that the theoretical multilevel model predicted the performance of children on the individual test-pairs rather well. keywords: fuzzy trace theory, Latent Gold, multilevel latent class model, transitive reasoning.

3 MLLC Analysis: an application to repeated transitive reasoning tasks 4 Multilevel Explanations The general aim of cognitive developmental research is the uncovering of relationships between cognitive processes, environmental influences and age (see e.g., Flavell, 1985; Siegler, 1991). Because cognitive processes can not be observed directly but only inferred from observable variables, one usually assumes that observable cognitive behavior is indicative for the underlying latent processes. Typically, observable behavior on a cognitive task can be explained by some kind of latent strategy or rule. Knowledge of the strategy used by a child enables a more precise prediction of the performance on a cognitive task. In turn, the (unobservable) strategy which is used might well be predicted by an underlying ability; that is, from a child s underlying ability level we can derive how likely it is that a child uses a particular strategy. Such a general explanatory model that assumes that cognitive behavior depends on strategy and that strategy depends on ability has the form of a hierarchical or multilevel model. This theoretical model can be tested by means of a multilevel latent class (MLLC) model; that is, by a model with two types of latent variables, ability and strategy, where strategy is a categorical latent variable at the task level and ability is a continuous latent variable at the child level. Various types of psychological phenomena can be described using this general theoretical model. For example, in the development of learning to read, it is common to distinguish three types of strategies, each of which corresponds to a particular developmental stage. These strategies might be called, spelling, context guessing, and complete reading. When using the type of multilevel explanation described above, we would assume that the likelihood of using these strategies depends on a child s reading ability the higher the ability

4 MLLC Analysis: an application to repeated transitive reasoning tasks 5 the more likely that strategy complete reading will be used and that the likelihood of performing a particular reading task correctly depends on the strategy used. For the performance on the balance scale task (Siegler, 1976) one may use a similar type of multilevel explanation. According to Siegler (see also Jansen & Van der Maas, 1997, 2002; Boom & ter Laak, in press) the performance on balance scale tasks depends on the specific rule that is used by the child. Siegler (1976) distinguishes four ordered rules. The probability that a particular rule is used can be assumed to be dependent on one or more underlying abilities. In the remainder of this article we describe an application of the MLLC model to transitive reasoning. We start with an introduction of the psychological theory a more detailed description of which was provided by Bouwmeeser, Vermunt, and Sijtsma (in press) and discuss the study design and instrument. Then we provide a more formal description of the employed MLLC model, as well as the results obtained in our study. Transitive Reasoning: Our Application An individual is capable of transitive reasoning if (s)he is able to infer an ordinal relationship between two objects from other ordinal relationships in which these objects are involved. For example, if one knows that stick A is longer than stick B, and that stick B is longer than stick C, the correct inference that stick A is longer than stick C gives evidence of transitive reasoning ability. In this example, the pairs [A, B] and [B, C] are the premise pairs and the relationships between the objects in the premise pairs constitute the premises. A transitive reasoning task consists of a presentation stage and a test stage. At the

5 MLLC Analysis: an application to repeated transitive reasoning tasks 6 presentation stage, the premise pairs are shown to the child. During the test stage, (s)he is asked to infer the transitive relationship from the premises. The object pair [A, C] is called the transitivity test-pair. The premise pairs [A, B] and [B, C] are called memory test-pairs. The latter are used to test whether the child is able to remember the premises tested. Figure 1 shows an example of a five-object transitivity task. During the presentation stage the child is confronted with the first four boxes. In the first box, the child compares sticks A and B and decides that A is the longer stick. In the second box the child decides that stick B is longer than stick C. In the third box the child decides that stick C is longer than stick D, and in the fourth box the child decides that stick D is longer than stick E. During the test-stage the child is first confronted with four memory test-pairs. Figure 1 shows the first memory test-pair [A,B]. In the test-stage the child can not observe the length differences anymore but (s)he has to remember or infer them. After the four memory test-pairs ([A,B],[B,C],[C,D],[D,E]) the child is confronted with three transitivity test-pairs ([A,C],[B,D],[C,E]). The recorded inferences for these seven test-pairs are the seven items corresponding to one task. Insert Figure 1 about here Fuzzy Trace Theory Brainerd and Kingma (1984, see also Brainerd & Reyna, 1993, 2004; Reyna & Brainerd, 1992, 1990, 1995a, 1995b) used fuzzy trace theory (FTT) to explain performance on memory and transitivity test-pairs. According to FTT children s performance on memory and

6 MLLC Analysis: an application to repeated transitive reasoning tasks 7 transitivity test-pairs can be explained by the use of gist traces and verbatim traces. Verbatim traces contain literal, detailed information of direct observable information. Gist traces contain reduced, pattern-like information. The use of both kinds of traces depends on the underlying verbatim and gist ability levels. According to FTT the performance on memory and transitivity test-pairs can be explained by the availability of relevant verbatim traces and gist trace. For memory test-pairs a child can use the observed information about the size of two sticks located on a verbatim trace. However, memory test-pairs can also be inferred by pattern information located on a gist trace. A gist trace may be, for example, objects get smaller to the left. This information can be used to infer that stick A is larger than stick B. Thus for memory test-pairs both verbatim traces and gist traces can be used. For transitivity test-pairs, however, a child needs pattern information located on a gist trace. According to FTT multiple traces process in parallel each containing different kind of information. For transitive reasoning, three verbatim traces and three gist traces are distinguished that lead to different responses on memory and transitivity test-pairs (Bouwmeeser et al., in press). Probabilities to retrieve and use verbatim trace and gist trace levels depend on verbatim and gist ability levels. The higher the ability level, the higher the likelihood of retrieving and accurately using the appropriate trace for a specific task and thus to produce correct answers for the items corresponding to that task. Verbatim Traces and Gist Traces Figure 2 shows the relationships between verbatim ability, verbatim trace and performance on memory test-pairs. Note that because the verbatim traces do not contain pattern infor-

7 MLLC Analysis: an application to repeated transitive reasoning tasks 8 mation, verbatim ability does not influence performance on transitivity test-pairs. The three levels in Figure 2 are connected by two probability structures: One connects ability levels with trace levels, and the other connects trace levels with memory test-pair performance. The multilevel structure may already be recognized in Figure 2. Insert Figure 2 about here Verbatim ability is hypothesized to induce verbatim traces according to a particular conditional probability structure. The probability distribution is defined as P (verbatim trace verbatim ability level), which is the probability of using a particular trace given a particular verbatim ability level. Note that both verbatim ability and verbatim traces are unobservable. We assume that there are three ordered trace levels (Bouwmeeser et al., in press). At the first level, the verbatim trace does not contain relevant premises. Instead of such information, for example about the length of sticks, this trace may contain information about the sticks color or shape. This lack of relevant information has the effect that children using this trace guess for the correct answer to each of the memory test-pairs ( guessing at the second level in Figure 2) and, as a result, performance probabilities are approximately at chance level. At the second level, the verbatim trace contains relevant but incomplete premise information. More specifically, we expect primacy and recency effects to lead to better performance on test-pairs for the premises presented first and last than on test-pairs for the premises presented in between ( temporal position in Figure 2). At the third level,

8 MLLC Analysis: an application to repeated transitive reasoning tasks 9 the verbatim trace contains all the relevant premises. This results in high performance probabilities on all memory test-pairs ( complete memory in Figure 2). The conditional probability structure is the following. It is hypothesized that P (guessing trace ability level) decreases as a function of ability (i.e., with increasing ability it is less likely that the guessing trace is retrieved), and is maximal when ability level is low; P (temporalposition trace ability level) first increases and then decreases as a function of ability and is maximal when the ability level is intermediate (i.e., the temporal-position trace is characteristic of intermediate ability levels, but rare for low and high levels); and P (completememory trace ability level) increases as a function of ability and is maximal when ability level is high (i.e., the complete-memory trace is most easily retrieved at high ability levels). These hypotheses are visualized in Figure 2: Solid arrows between the two latent variable levels indicate high probability, dashed arrows indicate lower probability, and dotted arrows indicate low probability. Figure 3 shows the relationships between gist ability, gist traces and performance on memory and transitivity test-pairs. FTT predicts that a gist trace affects performance on both memory and transitivity test-pairs (Brainerd & Kingma, 1984). One probability structure connects the ability and trace levels, and another connects the trace and performance levels. Insert Figure 3 about here

9 MLLC Analysis: an application to repeated transitive reasoning tasks 10 As with verbatim abilities and traces, it is hypothesized that gist ability induces gist traces according to a particular conditional probability structure. Three gist traces were hypothesized. Each gist trace corresponds with a particular probability to answer a memory or a transitivity test-pair correctly. At the first level, relevant pattern information is absent and children guess for the correct answer on all memory and transitivity test-pairs ( guessing in Figure 3). This is likely to result in poor performance. At the second level, the gist trace contains relevant pattern information but not the complete ordering. Bryant and Trabasso (1971) showed that when forming an internal representation, the end-anchored pairs are learned first followed by the middle pairs. This is a spatial-position effect ( spatial position in Figure 3). The correct performance probabilities are expected to be high for memory and transitivity test-pairs on both ends of the ordering, and lower for test-pairs in between. For example, when five objects are ordered as Y A < Y B < Y C < Y D < Y E and a child uses the spatial-position trace, good performance is expected on memory test-pairs [A, B] and [D, E] and transitivity testpairs [A, C] and [C, E]; and poorer performance is expected on the other test-pairs. At the third level, the trace contains the complete pattern information for reproducing the memory test-pairs and inferring the transitive relationships. The success probabilities on all memory and transitivity test-pairs are expected to be high. The performance on transitive reasoning tasks can be explained by the combination of each of the three verbatim trace levels and each of the three gist trace levels. This yields nine possible combinations, each of which induces typical performance on memory and transitivity test-pairs.

10 MLLC Analysis: an application to repeated transitive reasoning tasks 11 Transitive-Reasoning Tasks Children may differ in their verbatim and gist ability levels and this will likely result in different performance on transitive reasoning tasks. Transitive reasoning tasks may also vary in difficulty level depending on the complexity of the operations that have to be performed to infer the transitive relationship (Brainerd & Reyna, 1990). Task characteristics are expected to differentially influence the retrieval of verbatim traces and gist traces. For example, when objects in a task are positioned in a linear order and also presented in a linear order the cues on ordering are obvious. As a consequence, the required gist ability level is lower than when, for example, the objects are not positioned or presented in a linear order. We used three kinds of transitive reasoning tasks all having five sticks that differed in length (Y A > Y B > Y C > Y D > Y E, or Y A < Y B < Y C < Y D < Y E ). All tasks consisted of seven test-pairs, four of which were memory test-pairs ([A,B],[B,C],[C,D],[D,E]), and three transitivity test-pairs ([A,C], [B,D], [C,E]). The tasks differed with respect to the position of the sticks or the presentation of the sticks. They may be characterized as (1) ordered position, ordered presentation (O pos O pres ); (2) ordered position, disordered presentation (O pos D pres ); and (3) disordered position, ordered presentation (D pos O pres ). With each task type, first four premise pairs were presented before children were confronted with four memory test-pairs and three transitivity test-pairs. O pos O pres -tasks. Objects in O pos O pres tasks are ordered from small to large or from large to small. The presentation of the premises is also ordered. Thus, first premise pair [A, B] is presented, followed consecutively by premise pairs [B, C], [C, D], and [D, E]. Ordered presentation of ordered objects renders the use of pattern information

11 MLLC Analysis: an application to repeated transitive reasoning tasks 12 from gist traces rather easy. Figure 1 shows an example of an O pos O pres task. O pos D pres -tasks. In O pos D pres tasks, the objects are ordered from small to large or large to small. The presentation of the premise pairs is disordered; for example, first [C, D] is presented, followed consecutively by [A, B], [D, E], and [B, C]. The midterm relationships are always presented first and last, and the end anchors are always presented in between. Therefore, we are able to distinguish a temporal-position effect from a spatial-position effect in performance on the memory test-pairs (see also Brainerd & Kingma, 1984; Bouwmeeser et al., in press). D pos O pres -tasks. In D pos O pres tasks, the objects are positioned disordered. For example, stick A is in the third position in the box and stick B is in the first position, stick C is in the fifth position, stick D is in the second position and stick E is in the fourth position. The presentation of the premises is ordered. Thus, first premise pair [A, B] is presented, followed consecutively by premise pairs [B, C], [C, D], and [D, E]. Because positional cues about the ordering of the objects are not provided, a disordered position is expected to require both high verbatim and gist-ability levels. Consequently, not only the ordering has to be recognized but also the premises have to be memorized. Table 1 shows the expected performance patterns on the memory and transitivity testpairs for the three task types for the nine combinations of verbatim trace and gist trace levels.

12 MLLC Analysis: an application to repeated transitive reasoning tasks 13 Insert Table 1 about here The Adopted Multilevel Latent Class Model Figure 4 depicts the theoretical model used to explain individual differences in performance on memory- and transitivity test-pairs. At the lowest level of the hierarchy are the individual test-pairs (or items), which are 0/1 (incorrect/correct) response variables. In multilevel terminology, these are the level-one observational units. Test-pairs are nested within transitive reasoning tasks, where each task consists of seven pairs. The twelve 1 tasks performed by a child the repeated measures taken from a child are the level-two units. It is assumed that the activated latent traces during a task determines the probability of solving the seven test-pairs correctly. At the highest level of the hierarchy are the child s latent verbatim and gist abilities. More specifically, the children who are confronted with twelve task are the level-three units. The child-level abilities which are constant across tasks capture differences between children in the likelihood of using a particular latent verbatim trace and gist trace when performing one of the administered transitive reasoning tasks. Insert Figure 4 about here More formally, let test-pairs be indexed by k = 1,.., 7; tasks by i = 1,.., 12; and children by j = 1,.., N. Response variable Y ijk = 1 when child j gives a correct response to test-pair 1 Each child is confronted with four replications of each of the three task forms.

13 MLLC Analysis: an application to repeated transitive reasoning tasks 14 k in task i, and Y ijk = 0 otherwise. The 7 scores of child j on task i are collected in the vector Y ij, and all the 84 (12 7) scores of child j are collected in vector Y j. The MLLC model for response vector Y j consists of a lower-level and an upper-level part (Vermunt, 2003). The lower-level part is a rather standard latent class model for the 7 test-pairs belonging to a particular task (for response vector Y ij thus). The upper-level part has the form of a multilevel logistic regression model and connects the 12 tasks performed by a child. The lower-level part is a latent class model with two ordinal latent variables denoted by X ij and Q ij representing the verbatim traces and gist trace, respectively, for a particular task i. These two latent variables are assumed to have discrete realization between 0 and 1, with equal distances between categories: With three categories per dimension, x = 0.0, 0.5, or 1.0, and q = 0.0, 0.5 or 1.0. This yields a model with multiple ordinal latent variables that Magidson and Vermunt (2001) called a latent class factor model. The model for the response vector Y ij conditional on the activated traces has the following form 7 P (Y ij X ij = x, Q ij = q) = P (Y ijk X ij = x, Q ij = q). (1) k=1 This equation reveals the basic assumption of a latent class model: The scores on the seven test-pairs are mutually independent given the latent verbatim trace and gist trace levels of child j at task i. 2 The term P (Y ijk = 1 X ij = x, Q ij = q) represents the probability of a correct answer on test-pair k from task i given membership of class (x, q). These conditional response prob- 2 It should be noted if we do not impose constraints on the P (Y ijk = 1 X ij = x, Q ij = q), assuming again both X ij and Q ij has 3 categories, we have in fact a model that is equivalent to a latent class model with a single latent variable with 9 categories.

14 MLLC Analysis: an application to repeated transitive reasoning tasks 15 abilities can be used to understand typical performance of this latent class. The model for transitive reasoning (Figure 4) would provide a meaningful description of our data when the estimated probabilities of correct performance on test-pairs are similar in relative magnitude to the hypothesized probabilities (see table 1). This would mean that the model is able to explain individual differences in performance on memory and transitivity test-pairs due to the differential use of verbatim trace and gist trace levels. As shown by Magidson and Vermunt (2001), to account for the two-dimensional and ordinal nature of X ij and Q ij, the conditional response probabilities P (Y ijk X ij = x, Q ij = q) are parameterized as logistic models. More specifically, the probability of obtaining a correct response from child j on test-pair k of task i is restricted by a standard binary logistic regression model of the form P (Y ijk = 1 X ij = x, Q ij = q) = exp(β 0ki + β 1ki x + β 2ki q) 1 + exp(β 0ki + β 1ki x + β 2ki q), (2) where β 0ki is an intercept, and β 1ki and β 2ki are the main effects of verbatim trace level and gist trace level, respectively. The indices k and i indicate that these parameters differ across test-pairs and tasks. This is, however, not fully correct since parameters were restricted to be equal for all four replications of the same task-type (e.g., β 0k,i+3 = β 0k,i ). This implies that only three sets of free β parameters need to be estimated. The upper-level part of the model deals with the nesting of tasks within children; that is, with the fact that the standard assumption of independent observations does not hold for our data. The multiple tasks performed by a child can, however, be assumed to be mutually independent given the child s latent verbatim and gist abilities. These two continuous latent variables, which are denoted by W j and V j, with realization w and v, have the role of random

15 MLLC Analysis: an application to repeated transitive reasoning tasks 16 effects in the (logistic regression) models for X ij and Q ij (Vermunt, 2003). As was indicated in the text, these abilities or random effects W j and V j determine child j s probability of activating a certain verbatim trace and gist trace level, respectively, at a particular task i. These probabilities, which are denoted as P (X ij = x W j = w) and P (Q ij = q V j = v) are usually referred to as class probabilities. Suppose X ij has three categories (classes) and these classes have probabilities equal to.35,.40 and.25. Applied to our study, these probabilities would indicate that at each task child j has a probability of.35 to retrieve a low verbatim trace level (i.e., guessing trace ), of.40 to retrieve an intermediate verbatim trace level (i.e., temporal-position trace ), and of.25 to retrieve a high verbatim trace level (i.e., complete-memory trace ). Typical for the MLLC model is that these probabilities vary across higher-level units (children) whereas in a standard latent class model these probabilities are assumed to be the same for all cases. The relationships between W j and X ij and between V j and Q ij are parameterized by logistic regression models; that is, P (X ij = x W j = w) = exp(γ 0x + γ 1 x w) x exp(γ 0x + γ 1 x w), and P (Q ij = q V j = v) = exp(γ 2q + γ 3 q v) q exp(γ 2q + γ 3 q v). These are adjacent-category ordinal logistic regression models similar to the ones used in partial-credit models, which are item response models for ordinal items. The γ parameters are assumed to be equal across the 12 tasks. Using multilevel terminology, γ 0x (γ 2q ) is the fixed part in the regression model for X ij (Q ij ) and γ 1 (γ 3 ) is parameter associated with the random part of the model, where we can either estimate γ 1 (γ 3 ) and fix the variance

16 MLLC Analysis: an application to repeated transitive reasoning tasks 17 of random effect W j (V j ) to 1 or fix γ 1 (γ 3 ) to 1 and treat the variance of W j (V j ) as a free parameter. This upper-part of our MLLC model is similar to the mixed-effects logistic regression model for indirectly observed categorical variables discussed by Vermunt (2005). The only difference is that here we have two such models, one for X ij and one for Q ij. In Appendix A, we show how the lower- and upper parts are merged to obtain the complete MLLC model for P (Y j ). Maximum likelihood estimates of the parameters of the presented MLLC model can be obtained with Latent GOLD (Vermunt & Magidson, 2005), a user-friendly Windows-based program for latent class analysis that is available at The setup for running the presented MLLC model with the syntax version of Latent GOLD is provided in Appendix B. Alternative models Besides the hypothesized model described above (Model A), we tested a series of alternative models (Models B to J). First, we checked whether we really need the complex multilevel structure from Model A by comparing its fit with the fit of a series of simpler models (Models B to E). Model B contains no upper level (no abilities) and thus considers the performance of a child on one task independent of the performance on another task. Model C is a standard two-dimensional item-response model without mediating level (no traces) and serves as an alternative non-multilevel explanation for our data. In Model D both the upper and lower-level latent variables are omitted, implying that test-pairs are treated as independent observations. Model E is a simpler variant of our MLLC model containing only one upper- and one lower-level latent variable (one ability and one trace).

17 MLLC Analysis: an application to repeated transitive reasoning tasks 18 Second, we assessed whether the assumptions made in our MLLC model hold by comparing the results with a series of less restricted models each of relaxes one of its assumptions (Models F to I). In Model F we relieve the assumption that the verbatim traces and gist traces are ordered and treat trace levels a nominal categories. In Model G we allowed a correlation between the gist and verbatim ability (note that in the hypothesized model these abilities are assumed to be uncorrelated). In Model H we allowed an association between the ordered verbatim traces and gist traces. Model I relaxes the assumption that the effects of abilities on responses are fully mediated by the traces by including direct effects from abilities on test-pairs. Third, the validity of the identified abilities was investigated by determining the strength of their relationship with age (see also Boom & ter Laak, in press). More specifically, Model J contains the variable age-grade as a covariate for the two latent abilities. Method Instrument An individual computer test for transitive reasoning was constructed (Bouwmeester & Aalbers, 2004). Binary performance scores were registered automatically during test administration. Four test versions each presented the tasks in a different order. Sample The transitive reasoning test was administered to 409 children ranging from 5 to 13 years of age. Children came from four elementary schools in the Netherlands. They were from middle class social-economic status (SES) families.

18 MLLC Analysis: an application to repeated transitive reasoning tasks 19 Design Four versions of each task type were administered; thus, there were 12 tasks in total. The four tasks of the same type differed with respect to the colors of the sticks, and with respect to the direction of the ordering; A 1-score was assigned when the child touched the correct stick on the pc-screen; and a 0-score was assigned otherwise. For each child, 7 (test-pairs) 3 (task types) 4 (task-type versions) = 84 scores were collected. Procedure The test was administered in a quiet room in the school building. Three introductory tasks were presented in which it was explained that each time the child had to touch the longest stick. Next, the experimenter explained that there were 12 additional tasks and that the child had to try these tasks on her/his own. Results Assessing Model Fit Table 2 shows the fit results for Models A through J. To compare the fit of the models while accounting for the number of parameters we used the Bayesian information criterion (BIC). One of the terms in the BIC formula is the sample size, which can here be equated to either the number children (409) or the number of tasks performed by these children (4860). There is no agreement as to which variant should be used in multilevel analysis, but the second version is the more common one. We also used this more conservative variant. According to the BIC statistic, the hypothesized model performed better than any of the alternative Models B though I, except for model F. This indicates on the one hand that we really need a multilevel structure with two latent variables at both levels to describe the

19 MLLC Analysis: an application to repeated transitive reasoning tasks 20 data well, but on the other hand that our ordinal specification for the mediating level may be somewhat too restrictive. The fact that Model J performed better than Model A shows that the encountered latent abilities are significantly related to age. Insert Table 2 about here Reproduction of Probability Structure For all test-pairs, the estimated performance probabilities on the memory and transitivity test-pairs of Model A were compared with the expected performance probabilities (Table 1). In general, the hypothesized performance according to the verbatim and gist traces agrees with the estimated performance. That is, verbatim traces only influence the performance on memory test-pairs while gist traces influence the performance on both memory and transitivity test-pairs. Table 3 shows the hypothesized and the estimated performance probabilities of the testpairs of the four O pos O pres tasks. The majority of the estimated probability patterns agreed with the hypothesized patterns. However, in the fourth row the patterns of the hypothesized and the estimated probabilities differed for the memory test-pairs: It was hypothesized that intermediate verbatim trace level and low gist trace level produce a temporal-position effect. However, the high probabilities found suggest complete memory of the premises. Insert Table 3 about here

20 MLLC Analysis: an application to repeated transitive reasoning tasks 21 Table 3 shows that for the four O pos D pres tasks the majority of the estimated probability patterns agreed with the hypothesized patterns but that patterns differed in rows 2 and 4 (Task O pos D pres ). The estimated probabilities in row four showed that the spatial-position effect was only active on one end-anchor, which led to high probabilities for the test-pairs M 3 and T 3 and lower probabilities for the test-pairs M 2 and T 2. The estimated probabilities showed the temporal position effect only for the first memory test-pairs (M 1 and M 2 ). For the four D pos O pres tasks, Table 3 shows that four estimated probability patterns agreed with the hypothesized patterns (in rows 1, 2, 7, and 9). Five patterns (Task D pos O pres, in rows 3, 4, 5, 6, 8) were different. First, for low verbatim trace level and high gist trace level (Table 3, Task D pos O pres, row 3), for all test-pairs low probabilities were hypothesized. However, the estimated probabilities suggested a spatial-position effect which resulted in moderate and high success probabilities for the test-pairs M 1, M 4, T 1, and T 3. Second, the temporal position effect was only active on one side for rows 4, 5 and 6. Third, a spatial position effect was active for a combination of high verbatim trace and intermediate gist trace (row 8) while this was not expected. Discussion The use of the multilevel latent class model to transitive reasoning performance can be assumed to be an enrichment for the research on transitive reasoning. It enabled us to test detailed hypotheses making it possible to determine which aspects of fuzzy trace theory agreed with observed data and which aspects disagreed. Brainerd and Kingma (1984) described the consequences of fuzzy trace theory on transitive reasoning in great detail, how-

21 MLLC Analysis: an application to repeated transitive reasoning tasks 22 ever, they were not able to falsify the details of the theory by an analysis of variance designs with existing (age) groups and average performances on different kinds of tasks. Our results showed that two abilities fitted the data better than one ability and that these abilities could indeed be interpreted as verbatim and gist abilities. Moreover, the performance of children could well be explained by the combination of three verbatim trace and three gist trace levels. The estimated probabilities for the test-pairs of O pos O pres tasks and O pos D pres tasks in general agreed with the hypothesized probabilities. For D pos O pres tasks discrepancies were larger. The presented application of MLLC analysis was already a rather complicated one. While the models that were originally proposed by Vermunt (2003) contained a single (nominal or continuous) upper-level and a single nominal lower-level latent variable, in our application we used two continuous upper-level and two ordinal lower-level latent variables. In other words, we used a multidimensional extension of the models by Vermunt, as well as an extension to ordinal latent classes. Whereas our application involved the analysis of repeated measures experimental data, other possible applications of the MLLC model in developmental psychology include the analysis of longitudinal data and data with a natural group structure, such as data from family studies. Although the employed statistical model would be similar, the substantive interpretation of the results would be very different in such applications. With longitudinal data applications, our method could serve as an alternative to latent transition or latent Markov analysis (Collins & Wugalter, 1992; Vermunt, Langeheine, & Böckenholt, 1999; Schmittmann, Dolan, van der Maas, & Neale, 2006). While the latter type of analysis

22 MLLC Analysis: an application to repeated transitive reasoning tasks 23 focuses on describing and explaining (aggregate) transitions between time-specific latent states, the MLLC model yields a growth model for latent states. Such a model could be used to detect individual differences in development across children when the developmental stage is a discrete latent variable. In family studies, the MLLC model could be used to detect variations in class membership probabilities across families, which can usually be attributed to genetic and common environment effects. An application using a six continuous measures intelligence test was provided by Vermunt (in press). The multilevel approach can also be used with other types of grouped data, such are applications in which higher-level units are geographical regions, schools, medical centers, etc., where the aim of the study is to determine differences in class membership probabilities between these higher-level units. Appendix A: Merging the lower- and upper part of the MLLC model To merge the lower- and upper-level parts of our MLLC model, we first have to derive the probability of Y ij conditional on latent abilities W j and V j, P (Y ij W j = w, V j = v). This involves combining the probabilities P (Y ij X ij = x, Q ij = q), P (X ij = x W j = w), and P (Q ij = q V j = v) defined above as follows: P (Y ij W j = w, V j = v) = x P (X ij = x W j = w) P (Q ij = q V j = v)p (Y ij X ij = x, Q ij = q). q (3) As can be seen, X ij and Q ij are assumed to be mutually independent conditionally on W j

23 MLLC Analysis: an application to repeated transitive reasoning tasks 24 and V j. Moreover, the effects of the continuous latent abilities on the responses are assumed to be fully mediated by the discrete latent trace levels. Note that P (Y ij X ij = x, Q ij = q) is the probability defined in equation (1). The probability associated with all responses of an individual, denoted by P (Y j ), can now be obtained by taking the product of P (Y ij W j = w, V j = v) over the 12 tasks and integrating the two continuous latent ability variables out of the equation. This yields: P (Y j ) = w v f(w j = w) f(v j = v) [ 12 P (Y ij W j = w, V j = v) i=1 ] d w d v. (4) Note that P (Y ij W j = w, V j = v) has the form described in equation 3, and f(w j = w) and f(v j = v) are standard normal univariate distributions. Appendix B: Setup for Latent GOLD 5.0 Syntax The relevant MLLC models can easily be defined with the syntax version of Latent GOLD 5.0. For our application, we could either use a data set in the form of a univariate three-level data set or a multivariate two-level data set. We used the latter specification, in which the responses on the seven test-pairs are in seven columns of the data file labeled Y1 to Y7. Omitting the options section and the definition of the data file, this is the set up for the hypothesized model (Model A): variables groupid childid; dependent Y1, Y2, Y3, Y4, Y5, Y6, Y7; independent form nominal;

24 MLLC Analysis: an application to repeated transitive reasoning tasks 25 latent W continuous group, V continuous group, X ordinal 3, Q ordinal 3; equations X <- 1 + W; Q <- 1 + V; Y1 <- 1 form + X form + Q form; Y2 <- 1 form + X form + Q form; Y3 <- 1 form + X form + Q form; Y4 <- 1 form + X form + Q form; Y5 <- 1 form + X form + Q form; Y6 <- 1 form + X form + Q form; Y7 <- 1 form + X form + Q form; (1) W; (1) V; In the variables section we defined the seven response (dependent) variables. The variable form indicating the task version and taking on values 1, 2, and 3 is used as an independent variable. The childid variable connecting the 12 records (tasks) of a child is used as the groupid. The last part of the variables section defines the two continuous group-level (here child-level) latent variables (abilities) and the two ordinal task-level latent variables (traces),

25 MLLC Analysis: an application to repeated transitive reasoning tasks 26 each of which is assumed to have 3 categories. The equations section completes the definition of the model to be estimated. Basically, this section contains the logistic regression equations (3) and (2) for the two discrete latent variables and for the seven item responses, respectively. The equation for X (Q) contains an intercept (denoted by 1) and an effect of W (V). The regression models for the item responses contain an intercept, an effect of X, and an effect of Q, where the term form indicates that the value of the parameter concerned differs across (is conditional on) the 3 task forms. The terms (1) W and (1) V are variance equations indicating that the variances of W and V should be fixed to 1 (which is also the Latent GOLD default). This model setup can easily be modified to obtain the other models that were estimated. For example, changing the scale type of X and Q from ordinal into nominal yields Model F, adding the equation W V; yields the model with correlated abilities (Model G), and adding age-grade to the list on independent variables and equations V age-grade; and W age-grade; yields the model with covariate age-grade (Model J).

26 MLLC Analysis: an application to repeated transitive reasoning tasks 27 References Boom, J., & ter Laak, J. (in press). Classes in the balance: Latent class analysis and the balance scale task. Developmental Review. Bouwmeeser, S., Vermunt, J. K., & Sijtsma, K. (in press). Development and individual differences in transitive reasoning. Developmental Review. Bouwmeester, S., & Aalbers, T. (2004). Tranred 2. Tilburg: Tilburg University. Brainerd, C. J., & Kingma, J. (1984). Do children have to remember to reason? A fuzzytrace theory of transitivity development. Developmental Review, 4, Brainerd, C. J., & Reyna, V. F. (1990). Gist is the grist: Fuzzy-trace theory and the new intuitionism. Developmental Review, 10, Brainerd, C. J., & Reyna, V. F. (1993). Memory independence and memory interference in cognitive development. Psychological Review, 100, Brainerd, C. J., & Reyna, V. F. (2004). Fuzzy-trace theory and memory development. Developmental Review, 24, Bryant, P. E., & Trabasso, T. (1971). Transitive inferences and memory in young children. Nature, 232, Collins, L. M., & Wugalter, S. E. (1992). Latent class models for stage-sequential dynamic latent variables. Multivariate Behavioral Research, 27, Flavell, J. H. (1985). Cognitive development. Englewood Cliffs, NJ: Prentice-Hall, Inc.

27 MLLC Analysis: an application to repeated transitive reasoning tasks 28 Jansen, B. R. J., & Van der Maas, H. L. J. (1997). Statistical test of the rule assessment methodology by latent class analysis. Developmental Review, 17, Jansen, B. R. J., & Van der Maas, H. L. J. (2002). The development of children s rule use on the balance scale task. Journal of Experimental Child Psychology, 81, Magidson, J., & Vermunt, J. K. (2001). Latent class factor and cluster models, bi-plots, and related graphical displays. Sociological Methodology, 31, Reyna, V. F., & Brainerd, C. J. (1990). Fuzzy processing in transitivity development. Annals of Operations Research, 23, Reyna, V. F., & Brainerd, C. J. (1992). A fuzzy-trace theory of reasoning and remembering: paradoxes, patterns, and parallelism. In A. Healy, S. M. Kosslyn, & R. M. Shiffrin (Eds.), Essays in honor of William K. Estes vol. 2 (pp ). Hillsdale, NJ: Erlbaum. Reyna, V. F., & Brainerd, C. J. (1995a). Fuzzy-trace theory: An interim synthesis. Learning and Individual Differences, 7, Reyna, V. F., & Brainerd, C. J. (1995b). Fuzzy-trace theory: some foundational issues. Learning and Individual Differences, 7, Schmittmann, V. D., Dolan, C. V., van der Maas, H. L., & Neale, M. C. (2006). Discrete latent markov models for normally distributed response data. Multivariate Behavioral Research, 40,

28 MLLC Analysis: an application to repeated transitive reasoning tasks 29 Siegler, R. S. (1976). Three aspects of cognitive development. Cognitive Psychology, 8, Siegler, R. S. (1991). Children s thinking, second edition. New Jersey: Prentice-Hall,Inc. Vermunt, J. K. (2003). Multilevel latent class models. Sociological Methodology, 33, Vermunt, J. K. (2005). Mixed-effects logistic regression models for indirectly observed outcome variables. Multivariate Behavioral Research, 40, Vermunt, J. K. (in press). Latent class and finite mixture models for multilevel data sets. Statistical Methods in Medical Research. Vermunt, J. K., Langeheine, R., & Böckenholt, U. (1999). Discrete-time discrete-state latent markov models with time-constant and time-varying covariates. Journal of Educational and Behavioral Statistics, 24, Vermunt, J. K., & Magidson, J. (2005). Latent Gold 4.0. Belmont, MA: Statistical Innovations Inc.

29 MLLC Analysis: an application to repeated transitive reasoning tasks 30 Table 1. Expected Performance on the Test-Pairs for Nine Combinations of Trace Levels on three tasks Verbatim Gist Memory Transitivity M1 M2 M3 M4 T1 T2 T3 description low poor performance on memory- and transitivity test-pairs low intermediate good performance on memory- and transitivity test-pairs high good performance on memory- and transitivity test-pairs low temporal position effect for memory test-pairs poor performance on transitivity test-pairs Task intermediate intermediate good performance on memory- and transitivity test-pairs OposOpres high good performance on memory- and transitivity test-pairs low good performance on memory test-pairs and poor transitivity test-pairs high intermediate good performance on memory- and transitivity test-pairs high good performance on memory- and transitivity test-pairs low poor performance on memory- and transitivity test-pairs low intermediate spatial position effect for memory and transitivity test-pairs high good performance on memory- and transitivity test-pairs low temporal position effect for memory test-pairs poor performance on transitivity test-pairs Task intermediate intermediate temporal position effect and spatial position effect OposDpres high good performance on memory- and transitivity test-pairs low good performance on memory test-pairs and poor performance on transitivity test-pairs high intermediate good performance on memory test-pairs and spatial position effect on transitivity test-pairs high good performance on memory- and transitivity test-pairs low poor performance on memory- and transitivity test-pairs low intermediate poor performance on memory- and transitivity test-pairs high poor performance on memory- and transitivity test-pairs low temporal position effect for memory test-pairs poor performance on transitivity test-pairs Task intermediate intermediate temporal position effect for memory test-pairs poor performance on transitivity test-pairs DposOpres high temporal position effect and spatial position effect low good performance on memory test-pairs and poor performance on transitivity test-pairs high intermediate good performance on memory test-pairs and poor performance on transitivity test-pairs high good performance on memory- and transitivity test-pairs M1 =first memory test-pair; M2 =second memory test-pair; M3 =third memory test-pair; M4 =fourth memory test-pair; T1 =first transitivity test-pair; T2 =second transitivity test-pair; T3 =third transitivity test-pair; : poor performance; : moderate performance; : good performance

30 MLLC Analysis: an application to repeated transitive reasoning tasks 31 Table 2. Fit Measures for the Estimated Models Log-likehood Number of BIC Model Description Value Parameters Value A Hypothesized model B No abilities C No traces D No abilities and no traces E One ability and one trace F Nominal traces G Correlation between abilities H Association between traces I Incomplete mediation by traces J Age-grade affecting abilities BIC is an information criterion for comparing alternative models.

31 MLLC Analysis: an application to repeated transitive reasoning tasks 32 Table 3. Estimated Success Probability for the Test-Pairs for Nine Combinations of Latent Trace Levels Hypothesized probabilities Estimated probabilities Verbatim Gist Memory Transitivity Memory Transitivity M1 M2 M3 M4 T1 T2 T3 M1 M2 M3 M4 T1 T2 T3 row nr low low intermediate high low Task intermediate intermediate OposOpres high low high intermediate high low low intermediate high low Task intermediate intermediate OposDpres high low high intermediate high low low intermediate high low Task intermediate intermediate DposOpres high low high intermediate high : <.65; :.65.79; : >.79

32 MLLC Analysis: an application to repeated transitive reasoning tasks 33 Figure 1. Example of the premise presentation of an Ordered Position, Ordered Presentation task; Letters A,B,C,D, and E were not visible to the children. Note that the greys in fact were yellow, green, purple, red, orange, and blue when presented to the children.

33 MLLC Analysis: an application to repeated transitive reasoning tasks 34 Verbatim ability (Latent variable) Guessing Verbatim trace Temporal position Complete memory (Latent variable) Poor performance on all memory testpairs Performance on memory test-pairs Good performance on some test-pairs but poor on others Good performance on all memory testpairs (Manifest variable) Figure 2. Relationships between latent verbatim ability, latent verbatim traces, and performance on memory test-pairs.

34 MLLC Analysis: an application to repeated transitive reasoning tasks 35 Gist ability (Latent variable) Guessing Gist trace Spatial position Complete ordering (Latent variable) Performance on memory and transitivity test-pairs (Manifest variable) Poor performance on all test-pairs Good performance on some test-pairs but poor on others Good performance on all test-pairs Figure 3. Relationships between latent gist ability, latent gist trace, and performance on memory and transitivity test-pairs.

35 MLLC Analysis: an application to repeated transitive reasoning tasks 36 Figure 4. tasks. Individual difference model of fuzzy trace theory for three transitive reasoning