Using Latent Growth Modeling to Understand Longitudinal Effects in MIS Theory: A Primer

Transcription

1 Communications of the Association for Information Systems Using Latent Growth Modeling to Understand Longitudinal Effects in MIS Theory: A Primer Mark A. Serva Department of Accounting and MIS, The University of Delaware, servam@lerner.udel.edu Hemant Kher Department of Business Administration, The University of Delaware Jean-Philippe Laurenceau Department of Psychology, The University of Delaware Follow this and additional works at: Recommended Citation Serva, Mark A.; Kher, Hemant; and Laurenceau, Jean-Philippe (2011) "Using Latent Growth Modeling to Understand Longitudinal Effects in MIS Theory: A Primer," Communications of the Association for Information Systems: Vol. 28, Article 14. DOI: /1CAIS Available at: This material is brought to you by the AIS Journals at AIS Electronic Library (AISeL). It has been accepted for inclusion in Communications of the Association for Information Systems by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contact elibrary@aisnet.org.

2 Using Latent Growth Modeling to Understand Longitudinal Effects in MIS Theory: A Primer Mark A. Serva Department of Accounting and MIS, The University of Delaware ServaM@lerner.udel.edu Hemant Kher Department of Business Administration, The University of Delaware Jean-Philippe Laurenceau Department of Psychology, The University of Delaware The use of structural equation modeling (SEM) has grown dramatically in the field of management information systems (MIS) in the past twenty years, but SEM s focus has been primarily on cross-sectional data sets. Functionally, SEM has been used to test measurement and path models, but the SEM approach has not been applied to repeated measures designs. In this article, we describe latent growth models (LGMs), an extension of SEM, which focuses on how observed and/or latent variables change over time. The purpose of this paper is to provide a primer on the use of LGMs, as well as to advocate for its use to extend MIS theory. We illustrate several flexible applications of LGMs using longitudinal data, including conditional, unconditional, and dual growth models. We discuss the advantages of using LGMs over other more traditional longitudinal approaches, and highlight areas in MIS where researchers can use this technique effectively. Keywords: latent growth modeling, longitudinal analysis, structural equation modeling The manuscript was received 6/4/2009 and was with the authors 9 months for 3 revisions. Volume 28, Article 14, pp , March 2011

3 Using Latent Growth Modeling to Understand Longitudinal Effects in MIS Theory: A Primer I. INTRODUCTION Within the management information systems (MIS) discipline, structural equation modeling (SEM) has been a popular data analysis methodology over the past twenty years for examining models that consist of observable as well as non-observable (or latent) variables. SEM allows researchers to study hypothesized relationships between latent variables that are typically measured using two or more observable measures (i.e., manifest or indicator variables). SEM has, however, focused on the analysis of cross-sectional data. The purpose of this didactic study is to illustrate the use of a special type of SEM, called latent growth models (LGM) [Duncan et al., 2006] or latent curve modeling (LCM) [Bollen and Curran, 2006] for studying changes over time in observed or latent variables of interest to researchers in the field of MIS. Because of the widespread use of SEM within the MIS discipline, we believe that LGMs could open new lines of inquiry within the MIS discipline by enabling researchers to study longitudinal changes in constructs that have been otherwise investigated using cross-sectional or two point (before/after) designs. LGM is an effective longitudinal data analysis technique when the focus is on the study of change in individuals. An MIS researcher might be interested, for example, in understanding how computer anxiety (CA) changes in a group of undergraduate students over the duration of a semester in which they are enrolled in an introductory MIS course. In such a study, the researcher would be interested in answering several questions, perhaps starting with how CA changes over time that is, what is the form of this change for the entire group? To accomplish this goal, the researcher would identify a suitable measure for CA, and take repeated observations on this measure on the students over the semester. Unlike traditional longitudinal data analysis techniques, LGM allows researchers to make inferences about individual level effects as well as group effects. If the research objective is merely to determine how change in CA occurred in the entire group of students (i.e., interest is in group level effects), then using ordinary least squares to analyze data would suffice. But the researcher might also be interested in going beyond group level findings. For example, the researcher might hypothesize that the initial level of CA as well as the rate of change in CA varies (or differs) for each student. In other words, there are inter-individual differences [e.g. Willett and Sayer, 1994] with respect to parameters of change (i.e., intercept and slope) in CA. The interesting questions here would be whether these differences are significant and if so can these individual differences be explained systematically using one or more predictors? For example, do males with low initial levels of CA grow at a faster rate than males with high initial levels of CA? Is the rate of change in CA faster in females than males over the semester? Does the rate of change in one construct (e.g., instructor supportiveness) predict the rate of change in another construct (e.g., computer anxiety)? Addressing these questions necessitates the use of LGM (or similar methods, such as hierarchical linear modeling). The purpose of this article, therefore, is to introduce LGM and provide a primer on its use. We note the advantages of using LGMs over other more traditional longitudinal approaches and discuss the types of research questions that can be answered using the technique. We also discuss data and sample size requirements, model identification considerations, estimation methods, and model fit assessment statistics. We illustrate the application of LGM, using actual longitudinal data on computer anxiety (CA) collected from students in an introductory MIS course, and conclude by highlighting opportunities in MIS where researchers can use LGMs effectively in the future. II. BACKGROUND ON LATENT GROWTH MODELING Traditional longitudinal data analysis methods, such as repeated measures ANOVA (RANOVA), suffer from two major limitations in analyzing repeated measures data: these methods operate at the aggregate (group) level, and the assumptions inherent in them are rarely satisfied in practice. To explain how this happens, Figure 1 depicts trajectories of computer anxiety (CA) recorded for four different students taken at four equally spaced points in time in a semester. The mean trajectory is also listed. Model parameters derived using traditional longitudinal data analysis methods (including RANOVA) imply that the intercept (initial level of CA) and slope (rate of decline for CA) are consistent across the sample. The plot in Figure 1 shows, however, that neither implication is true. Each student s trajectory differs greatly from the mean trajectory one has a positive slope, one is relatively flat, and the other two decline rapidly. Thus, other than demonstrating that the sample on average experiences linear declines in CA over time, the derived parameters and 214

4 aggregate method provide very limited insight into the data. Data in Figure 1 also do not satisfy the compound symmetry assumption inherent in RANOVA which posits that variances across time are equal and that covariances are zero. A close inspection of data in Figure 1 shows that CA observations are spread out the most initially, and the spread in these observations decreases between time periods 2 and 3, violating the compound symmetry assumption. Figure 1. Computer Anxiety Trajectories for Four Students LGM, which operates within the SEM framework, takes a different approach to analyzing data presented in Figure 1. Assuming that participants experience linear changes in CA over time, the researcher would fit a LGM that captures linear changes to such data (see Figure 2a). Following the path diagram convention used to represent SEMs, circles or ellipses represent latent constructs and rectangular boxes represent observed variables. Thus in Figure 2a, T1- CA through T4-CA represent CA levels measured at four different points in the semester respectively, while the hypothesized model s parameters (i.e., the intercept and slope) are modeled as latent constructs. The paths from the intercept construct to all measured variables are fixed to 1, whereas the paths from the slope construct to the measured variables are fixed from 0 to 1 in equal increments of to model the hypothesized linear change. The derived means of the intercept and slope constructs represent the fixed (i.e. group) effects of the model, whereas the derived variances indicate the extent that individuals within the sample have different starting points (intercepts) and rates of change (slopes). Significant latent construct variances indicate the presence of interindividual differences in the slope and intercept, providing evidence that all observations do not follow the same trajectory. The two-headed arrow between the intercept and slope constructs allows these parameters to co-vary. A significant negative covariance between slope and intercept indicates that lower starting points result in higher rates of change. The model depicted in Figure 2a is referred to as an unconditional LGM. If significant inter-individual differences are found, the researcher next fits a conditional LGM, which includes one or more external predictor variables that are posited to explain the inter-individual variability. The predictor can be a continuous variable (e.g., extent of prior experience with computing), dummy variable (e.g., gender), or even another latent construct (e.g., instructor supportiveness). The conditional LGM shown in Figure 2b includes gender and instructor as dummy predictor variables. A significant path from gender to the intercept and/or slope constructs, for example, would indicate that males and females in the sample have different initial levels of CA and different rates of change in CA over time. In addition to opening new lines of inquiry, LGM has a number of more pragmatic advantages. RANOVA cannot handle missing observations on individuals. In practice, this limitation results in the removal of the entire observation, even if only one instance of the repeated measure is missing. This restriction can substantially reduce a researcher s effective sample size. As an SEM method, LGM can be used even when subjects have missing observations for one or more measurement occasions (although too many missing data points can impair the stability of the results) [Singer and Willett, 2003]. Although RANOVA requires the time interval between repeated measurement occasions to be equal, LGM data collection phases can occur either at equally spaced or at irregular time intervals by adjusting 215

5 the fixed factor loadings between the slope and the manifest indicators. Finally, LGM can handle an unbalanced design (i.e., where the design of the study creates different numbers of repeated measures for different subjects in the samples); RANOVA cannot be used to analyze data from unbalanced designs. Figures 2a and 2b. Path Diagrams for an Unconditional LGM (2a) and a Conditional LGM (2b) Data Collection Considerations for LGMs Requirements for using LGM are similar to other longitudinal analysis techniques [Singer and Willett, 2003]. First, repeated measurements on the focal variable must be collected from each individual. 1 The focal variable can be an observed variable or a latent construct. 2 To achieve an identified model, at least three repeated observations must be taken. Bollen and Curran [2006] provide the following minimum data requirements, which depend on the form of change under investigation. For fitting models for linear change, at least three repeated measures must be collected. For more complex shapes of change (such as quadratic or cubic trajectories) a minimum of four or five repeated measures (respectively) are required. The number of repeated measurements also impact model identification: see Appendix A for more information on model identification and calculation of model degrees of freedom. In general, having more repeated measures serves to increase the reliability for the estimation of the parameters of change [Singer and Willett, 2003]. A third feature of data collection design is to ensure that when repeated measures are collected, the same items 3 are used to measure the focal variable in which change is being investigated. Finally, a sensible metric for time should be chosen. In many cases time may be the appropriate metric between successive data collection occasions, while in other cases, number of repetitions of a particular task may be the appropriate metric between successive data collection waves. LGMs also tend to be quite efficient with regards to the required sample size. Muthen and Muthen [2002] provide guidelines on the influence of sample size on the statistical power of LGMs. Their results indicate that for unconditional models, even small samples (n = 40) can provide a statistical power of 0.80 or better. The presence of predictor variables in conditional LGMs increases the recommended sample size to about 150. Sample size requirements increase further in the presence of missing data and smaller effect sizes (e.g., small values for population regression coefficient). Sample size can also affect an LGM s propensity to converge on a solution. Previous studies indicate that sample sizes of 100 or more increase the convergence rate to nearly 100 percent while decreasing the chances of generating improper solutions [Hamilton et al., 2003] The cohort sequential data collection design allows researchers to relax the requirement of collecting data on the same individuals over the entire duration of the study [Duncan et al. 1996]. When the focal variable is a latent construct, a second order LGM is required [Hancock et al., 2001]. In some cases a researcher may use an instrument that contains parallel items rather than the exact same items in repeated administrations. 216

6 III. ILLUSTRATION OF LATENT GROWTH MODELING In this primer on LGM, we study changes in computer anxiety (CA) in students enrolled in an introductory MIS course at a large university. We explore three guiding questions that have become a standard approach for using the LGM methodology [Bollen and Curran, 2006]. For the first question, we examine how a construct of interest (i.e., computer anxiety) changes over time (e.g., does CA change in a linear or nonlinear manner). For the second question, we examine whether there are statistically significant inter-individual differences with respect to the parameters of change in our construct of interest (i.e., Do students start at different initial levels of CA? Is the rate of change for CA different for each student?). Finally, for the third question, if inter-individual differences do exist, what factors can be used to predict these differing initial levels and rates of change (i.e., Do men and women differ with respect to initial levels of CA as well as the rate of change?). Because two different instructors taught the introductory MIS course, we also include instructor as a control. The first two questions will be answered using an unconditional LGM (Figure 2a), while the third question requires the use of a conditional LGM (Figure 2b). From a theoretical standpoint, the following hypotheses will be addressed by the unconditional and conditional LGMs: H1: CA will decline in all students. H2a: There will be differences in the initial level of CA across students. H2b: There will be differences in the rate of change in CA across students. H2c: Students with higher initial level of CA will experience a faster decline in CA. H3a: Female students will exhibit higher initial levels of CA than males. H3b: Female students will exhibit a faster rate of decline in CA than males. Hypotheses H1, H2a, H2b and H2c can be addressed with the unconditional LGM (Figure 2a), while the conditional LGM (Figure 2b) with gender as a dichotomous predictor can address hypotheses H3a and H3b. We incorporate the data collection design features recommended by Singer and Willett [2003] in our study. We use data from four repeated observations on the same group of undergraduate students enrolled in an introductory MIS course over the semester. All scale items are listed in Appendix B. The first wave of data collection was administered at the start of the semester, and subsequent waves were evenly spaced over the course of the semester. We measured CA using the same four items used by Thatcher and Perrewe [2003] at every measurement occasion (Appendix B). Finally, the sample size (n = 230 students) in our study exceeds the minimum requirements stated in Muthen and Muthen [2002] and Hamilton et al. [2003] to fit the requisite LGMs that address our research questions. While attrition is a concern in collecting repeated measures data, a classroom setting allows for a more controlled environment in which to collect longitudinal data. Students were given an incentive in the form of bonus points for their participation. As a result, 75 percent of the students provided data at three or all four occasions. As recommended by Bollen and Curran [2006] and other researchers, we used the full information maximum likelihood (FIML) estimator to estimate the LGMs in the presence of missing data. Descriptive statistics for all measures are listed in Appendix C. Preliminary Data Analysis Latent growth modeling allows researchers to pursue either exploratory or confirmatory approaches to model a variable s functional form. The functional form refers to the variable s mean shape or trajectory, and can be linear, quadratic, or exponential, among others. For confirmatory approaches, sound theoretical understanding of the nature of change being investigated may guide the a priori selection of the functional form. As an example, in an industrial setting, shop floor personnel learning to perform a new task may be expected to reduce time to perform the task in accordance with an exponential curve (e.g., a growth or learning curve model). In such cases the researcher may use LGMs to confirm that the data conform to an exponential pattern. 4 Such an approach is analogous to using confirmatory factor analysis to test that specific questions load as expected on a pre-defined construct. Alternatively, LGM can be used as an exploratory technique in situations when prior research lacks guidance for the functional form of change for the phenomenon under investigation. Regardless of whether a confirmatory or an exploratory approach is used, once repeated measures have been collected, the norm is to use data plots to identify (or confirm) the functional form of change in the focal variable [e.g., Willett and Sayer, 1994]. Raw data on average CA for female students, male students, as well as all students is shown in Figure 3. The plot suggests that average CA declined linearly across all students. 4 For more information on specifying non-linear (e.g., quadratic or exponential) LGMs, see Bollen and Curran [2006]. 217

7 5 4 Female 3 Male Average 2 1 T1 T2 T3 T4 Figure 3. Mean Plots for Changes in Computer Anxiety for Female, Male, and All Students Selecting the Best Fitting Unconditional Growth Model To address the first guiding question, Chan and Schmitt [2000] suggest an approach for selecting the unconditional LGM that best describes the functional form of change in the focal variable or construct. The suggested approach involves comparing three possible trajectories for CA: no-growth [Stoolmiller, 1994], free form, and the linear models. The no-growth model contains only the intercept as a latent construct, with a loading of 1.0 to each measured time point. A good fit for this model would support the hypothesis that CA remains stable over time. In the free form and linear model, two latent constructs are used to model the intercept and slope parameters of the growth trajectory. For the free form model, the loadings from the intercept construct to all measured time points are fixed to one. The loadings from the slope construct to the first time point is fixed to 0; the loading from the slope to the last time point is fixed to 1. The loadings from the remaining time points (T2 and T3) to the slope are freed and are estimated by the SEM program, thus allowing for nonlinear shapes of change. The derived loadings for the second and third time point reflect the percentage growth that has occurred at time periods T2 and T3. Thus, if the estimated loadings for periods 2 and 3 are 0.30 and 0.70, the researcher would interpret that the focal variable underwent a 30 percent change between the first and second measurement, a further 40 percent change between the second and third measurement, and a 30 percent change between the last two measurements. The linear model is illustrated in Figure 2a; the fixed loadings for CA measured at time points T1, T2, T3 and T4 are 0, 0.333, 0.667, and 1 respectively. All models in this primer were fitted using the SEM software Mplus, version 4.21, but other SEM tools (such as Lisrel, EQS, or AMOS) can also be used. AMOS version 18 actually includes a growth curve model plug-in which automatically draws an LGM, given a specified number of time periods. All tools use the maximum likelihood (ML) approach for estimating LGMs, which is one of the most commonly used approaches [Bollen and Curran, 2006]. Mplus code for all LGMs in this primer can be found in Appendix D. The metrics used to assess model fit for LGMs are the same as those commonly used with SEM. Specifically, measures such as the chi-squared test statistic, goodness of fit index (GFI), adjusted goodness of fit index (AGFI), normed fit index (NFI), non-normed fit index (NNFI), comparative fit index (CFI), root mean squared error of approximation (RMSEA), and root mean squared residual (RMR or SRMR) are routinely reported in the context of assessing model fit. Researchers using SEM within the MIS discipline research have reported chi square, CFI, RMSEA and SRMR to describe goodness of model fit [Gefen et al., 2000]; we follow their lead and report these four measures for all LGMs in this study. To determine whether the no-growth, free form, or linear model represents the best fit, we compare nested models using the likelihood ratio test (also known as the chi-square difference test). In our example, the no-growth model is nested in the linear model, and the linear model is nested within the free form model. The chi-square difference test is implemented by calculating the significance level for the difference in model chi square values as well as the degrees of freedom for a pair of nested models. Table 1 illustrates model fit measures that includes RMSEA, SRMR, 218

8 and CFI. For the first two measures, values close to 0 indicate a good fit, while values greater than 0.10 are taken to imply a poor fit. For CFI, values over 0.90 indicate good fit. Fit statistics for the no-growth model are quite poor, while those for the linear and free form model are quite good. The chi-square difference test between the no-growth model and the linear model (Δχ 2 (3) = 42.29; p < 0.001) indicates the linear form model fits significantly better than the nogrowth model. The chi-square difference between the linear and free form models is not significant (Δχ 2 (2) = 0.418; p = 0.811) indicating that both the free form and linear model provide an adequate fit to changes in CA in the sample. The more parsimonious linear model is preferred for describing changes in CA in our sample (more parsimonious means more constrained that is, the model with greater degrees of freedom in our case). In conclusion, the LGM indicates that CA declines linearly. This conclusion is supported by the plots of average CA for males and females in Figure 3 which show that CA declined linearly over the semester. Table 1: Fit Statistics for Unconditional LGM for Computer Anxiety (CA) No-growth model Linear model Free-form model Chi-square Degrees of freedom P-value < RMSEA SRMR CFI The second question addresses the presence of significant inter-individual differences in starting point and slope for changes in CA. For the CA linear model, the intercept is 2.65 and the z-score is significant (35.64; p < 0.001), indicating that on average, students CA levels start out greater than zero. The z-score reflects the results from a standard z-test. The slope value of (z = -5.43; p < 0.001) indicates a significant decline in CA in each time period. The unconditional linear LGM is depicted in Figure 4. Figure 4. Unconditional Linear LGM The second guiding question addresses the presence of inter-individual differences. The Mplus output indicates that the variances for the intercept (z = 8.45; p < 0.001) and slope (z = 2.61; p = 0.009) are also significant. H1 is corroborated. The presence of a significant variance for the slope and intercept latent construct indicates that the mean intercept and slope for the sample does not reflect the sample as a whole. In practice, this result indicates that all students do not follow the same trajectory. These two variance levels indicate that student growth trajectories exhibit significant individual differences across the sample that is, that they differ from the mean initial CA level and 219

9 the mean CA decline rate. H2a and H2b are supported. The covariance between the intercept and slope is negative (-0.14), but it is not significant (z = ; p > 0.10). No support was found, therefore, for H2c. A significant negative covariance between the intercept and slope constructs would indicate that students with high initial levels of CA experienced a lower rate of decline (i.e., slope) in CA over time and vice versa. Incorporating Predictors of Change in LGMs Given the significant inter-individual differences in the intercept and slope for change trajectories in CA, we next explore the third guiding question by introducing exogenous factors (predictors) into LGMs that are hypothesized to explain the inter-individual differences found in the second step. A conditional model for change in CA was fit by including dummy variables for (1) gender (set to zero for females and one for males) and (2) a control for the two instructors (arbitrarily set to zero for one instructor and one for the other). The conditional model is shown in Figure 5. The conditional model for linear change in CA showed good fit (χ 2 (df = 9) = 6.524, p = , CFI = 1.00, RMSEA = and SRMR = 0.025). Paths from instructor to the two growth constructs were not significant, indicating that CA initial levels (0.13) and rate of change (-0.15) did not differ across two instructors. For gender, however, the path to the intercept of the growth model for CA was negative (-0.59; z = ) and statistically significant (p < 0.001), while the path to the slope was not significant (0.13; p = 0.428). As the gender dummy variable was coded 1 for men, our results indicate that men started the class with lower CA levels than women, but gender did not influence the rate of decline in CA 5 (i.e., the slope or the rate of reduction in CA is the same for males and females). This result, therefore, supports H3a, but not H3b. The parallel trajectories for average CA for male and female students shown in Figure 3 are consistent with these findings. Figure 5. Conditional Linear LGM with Gender and Instructor as Predictor Variables Dual Growth Model As a final application of LGMs, we illustrate a dual growth model, which is an LGM that links growth in two different focal variables simultaneously. Such models allow researchers to detect, for example, whether the trajectory parameters (e.g., intercept and slope) in an exogenous construct are related to the trajectory parameters (e.g., intercept and slope) in an endogenous construct. Such a model cannot be completed using RANOVA or even hierarchical linear modeling: it is only currently possible with LGM. The path diagram in Figure 6 shows that the intercept of ISP is hypothesized to impact the intercept and slope of CA. We also hypothesize that the rate of change in ISP is expected to affect the rate of change in CA (since the rate of change in ISP cannot logically affect 5 One may consider this to be a contradictory result given that the unconditional model for CA indicated significant variance in the slope construct. However, the proper interpretation here is that while there is a significant variance in the slope construct for the decline in CA, gender does not explain the differences in slopes across students, and that some other predictor might do so. 220

10 the initial rate of CA, this path is omitted). We retained gender and instructor as the predictors of change trajectories in CA. We also included a distal outcome 6 variable, end of semester computer self-efficacy (CSE), in our model. The path diagram in Figure 4 shows that growth constructs (i.e., slopes and intercepts) for CA and ISP are modeled to influence the CSE in students. In creating the ISP scale, we used two questions from an organizational supportiveness scale [Eisenberger et al., 1986, Appendix B]. The wording of the questions ( The instructor cares about my general satisfaction with the course and The instructor takes pride in my accomplishments in the course ) were adapted to fit our context. We measured each of the ISP items at the same time as CA over four time periods. Figure 6. A Dual Growth LGM with Predictors and a Distal Outcome A variety of instruments have been used within the MIS literature to measure CSE. Because the course in this study focused on IT applications (e.g., spreadsheets, database, HTML) as well as understanding of specific computing concepts, we used the task-specific instrument to measure CSE. A review of CSE instruments [Marakas et al., 1998], as well as a 2007 study [Downey and McMurtrey, 2007] supports the use of task-specific CSE instrument over the more general CSE instrument [Compeau et al., 1999]. We followed the recommendations of a previous study [Marakas et al., 1998] to develop measures of task-specific CSE. First, a set of representative tasks for each of the course modules were determined by the two course instructors. Next, we asked each instructor to list separately the most critical skills the students needed to perform within each module. The instructors then reviewed each other s lists and reconciled the differences. After these steps, the instructors did not otherwise participate in the study. These tasks were then included on the survey, which asked students to rate their level of expertise for a variety of representative tasks for each of the course modules. An example question for the spreadsheet module was Using a spreadsheet package, I believe I have the ability to properly use relative and absolute addressing. Students first indicated if they had the skill (Yes/No). If Yes, they then indicated their confidence (10% 100%) that they could complete the task. If No, a zero was recorded as the response. The indicator for CSE at the end of the semester was the student s total CSE score across HTML, database, spreadsheet, and general computer knowledge. All questions are listed in Appendix B. 6 A distal outcome, also known as a sequela [Duncan et al., 2006], is an outcome measure affected by the change in the focal variable. 221

11 The Mplus code for this model is provided in Appendix D. The fit statistics for the dual growth model in Figure 4 are fairly good (χ 2 = , df = 37, p = , CFI = 0.995, RMSEA = 0.020, and SRMR = 0.042). The mean for the intercept and slope of linear growth in ISP were statistically significant (mean intercept = 4.91, p < 0.001, and mean slope = 0.40, p < 0.001), suggesting that students perceived instructor support grew linearly over the semester. Similarly, the variances for these constructs were also significant (p < for both) suggesting that the growth trajectories were different for each individual. The covariance between the intercept and slope is negative and significant (-0.20; p = 0.025) suggesting that students with lower initial ISP experienced a faster rate of growth in ISP over the semester and vice versa. As with the conditional model results presented earlier, gender is a significant predictor for the initial level of CA but not for the rate of decline in CA over the semester, while the instructor did not influence change trajectories in CA. For the dual growth relationships between CA and ISP, only the path from the intercept of ISP to the intercept of CA is significant (loading = , p = 0.064). 7 This negative association suggests that students with high initial perceptions of instructor support had lower initial levels of CA. Had other paths been significant, they would have been interpreted similarly as to the impact of growth constructs for ISP on the growth constructs for CA. For the four paths between the CA and ISP growth parameters and the distal outcome, three are significant. The path from the intercept of CA on CSE is significant and negative (loading = , p < 0.001) suggesting that students with lower initial levels of CA have higher end of semester CSE. Paths from the intercept and slope of ISP to CSE are significant and positive (4.26, 4.673, p = 0.007, and p < respectively). The path between the slope of CA and the distal outcome was insignificant. These results suggest that higher initial levels of ISP and low initial levels of CA as well as a higher rate of growth in ISP lead to a higher end of semester CSE. IV. A SUMMARY FOR APPLYING LATENT GROWTH MODELING IN MIS We summarize the application of LGM as a method of analysis to longitudinal repeated measures data for studying change in focal variables of interest for MIS researchers as a series of steps below. Step 1: The first step involves proposing a hypothesis (or hypotheses) regarding the valence of longitudinal change in the focal variable (or variables) of interest. In this primer the focal variable was computer anxiety (CA). If theory provides guidelines, then the form of expected change in the focal variable should also be specified. LGM allows change to be specified in a variety of functional forms, including exponential, quadratic, cubic, and free form (see Bollen and Curren, 2006, for more details). When theory provides guidance, researchers should also hypothesize whether the constructs are expected to exhibit inter-individual differences (i.e., that the described change is expected to vary across groups of individuals). Prior studies have used two point designs to show that CA declines over time [Buche et al., 2007; Compeau et al., 1999], but the nature of the decline is unknown. Of the possible hypotheses that we suggested earlier in the manuscript, support was found for all but the last (H3b). Step 2: In Step 2, the researcher decides the appropriate strategy regarding the survey instrument (use an existing instrument, or create a new one). Variables that might explain possible inter-individual differences (e.g., gender, socio-economic status) should also be identified for use in the conditional models. For this primer, we measured CA at each data collection wave using the same four items that have been used in prior studies. Similarly, we measured perceived instructor support (ISP) using the same items at every measurement occasion. We also recorded each student s gender, as well as the instructor who taught the course. Step 3: The researcher must select the appropriate sample i.e., the group of individuals that will experience change with respect to the focal variables. The sample should be sufficiently large to permit investigation of the hypothesized forms of change. In this paper our sample consisted of undergraduate students enrolled in the introductory MIS class within a business school. Our sample (n = 230) was large enough to allow us to fit both unconditional and conditional LGMs. Step 4: The researcher must decide on the duration of the longitudinal investigation along with number of repeated administrations of the instrument that captures the focal variable of interest. The number of repeated measurements of the focal variable dictates the types of functional forms of change that can be investigated: with only three repeated measures, for example, only a linear form of change can be verified. For complex, nonlinear forms of 7 Proper interpretation of these results is important to understand the advantages of LGM over other statistical approaches. The added value of LGM in a dual growth model can perhaps best be illustrated when the exogenous construct s slope is significantly related to the endogenous construct s slope. In that situation, change in the exogenous construct is related to change in the endogenous construct. Such a finding is analogous to finding a significant relationship between two slopes, which of course cannot be completed using any other traditional statistical procedure, such as RANOVA or OLS regression. 222

12 change, four or more observations may be necessary (see Appendix A). In our study the duration of data collection spanned a semester. We measured CA at four equally spaced intervals, which would more than suffice if CA changed linearly. If CA s trajectory was nonlinear, however, at least four observations would be required to fit a quadratic change LGM. Step 5: Once data have been collected, researchers should prepare plots of the focal variables to ascertain (or confirm) the functional form of change. As illustrated in Figure 1, CA appeared to exhibit a linear decline over the course of the semester, providing insight into the appropriate unconditional LGM model. 8 Step 6: The researcher fits LGMs consistent with the hypothesized functional form of change, much as one fits a confirmatory factor analysis (CFA) model. An exploratory approach is also possible whereby competing models with alternate functional forms of change can be compared. We demonstrated the exploratory approach by fitting three unconditional models (no-growth, linear growth, and free form). The no-growth model demonstrated a poor fit and was eliminated. Both linear and free form models provided excellent fit. Based on the raw data plots in Figure 3 and on the parsimony of model constraints, we concluded that the unconditional linear LGM provided the best fit to decline in CA in our sample. Step 7: After identifying unconditional LGMs that best describe the functional form of change in the focal variable, the researcher provides and interprets statistics related to the constructs that describe the change. Specifically, with respect to linear change in CA, statistically significant values for the mean intercept and slope suggested a non-zero intercept and a negative slope. Further, statistically significant variances for the intercept and the slope construct provided evidence of inter-individual variation with regards to trajectories of CA change across students. The covariance between these constructs was not significant in our case; when the covariance is significant and negative, higher values of the intercept generally have lower values of slope and vice versa. Step 8: Once evidence of inter-individual variability with regard to the constructs of growth is found, the researcher proceeds to fit conditional LGMs that include predictors of change. We illustrated conditional growth models with two different predictors gender and instructor. Our results showed that growth constructs describing decline in CA did not vary across the two instructors. With regard to gender, our results showed that only the intercept of CA was different for males and females; specifically the intercept was lower for males. The rate of decline in CA was identical for both groups. The parallel trajectories for average CA for females and males in Figure 3 substantiate these findings from the conditional LGM. Step 9: Based on the hypotheses proposed in Step 1, the researcher can next explore more advanced models, which can provide insight into how the trajectory of one construct or variable relates to the trajectory of another construct or variable. We fitted a dual growth model (also known as parallel growth or cross domain growth) linking the trajectory of CA with the trajectory of perceived instructor support (ISP). The intercept and slope of the growth function for ISP were hypothesized to influence the intercept and slope of decline in CA. In this model, we retained gender and instructor as predictors of CA trajectories, and included end of semester computer self-efficacy (CSE) as a distal outcome of the intercept and slope for changes in CA and ISP. We note that a growth model that simultaneously includes predictors and distal outcomes is not possible with current version of RANOVA or even hierarchical linear modeling [Duncan et al., 2006]. This model showed that initial CA levels, initial ISP, and the rate of growth in ISP significantly influence end of semester CSE. V. DISCUSSION In a discipline that changes as rapidly as MIS, tracking a construct s evolution over time may be critical for understanding its role in the broader nomological network [Pinsonneault and Kraemer, 1993]. Many of the constructs commonly used in MIS research have an implied longitudinal dimension, but are primarily studied at one point in time. Turnover, for example, is a common concern in MIS organizational research, but is rarely examined for systematic fluctuations over time. Although turnover cannot be examined as a repeated measure, the trajectory for voluntary employee withdrawal [Wright and Bonett, 1993] as well as its precursors and consequences could offer a rich area for future research. Within TAM research, the effects of perceived ease of use and usefulness are likely to vary over time, since the role that a system plays in an organization tends to evolve. As a user learns the features of a technology, for example, the effects of these constructs are likely to fluctuate. One can conceive that within different theoretical contexts, the effect of usefulness and ease of use may increase or decrease over time 8 For a more complete discussion of the differences between theory-driven and exploratory research approaches, see Gurbaxani and Mendelson [1990]; Collopy et al. [1994]; Gurbaxani and Mendelson [1994]. When using an exploratory approach, researchers can use the plots to suggest an appropriate subsequent line of analysis. As with all exploratory approaches, however, the researcher is expected to disclose that the approach is theory-building not confirmatory and, therefore, requires subsequent confirmation. 223

13 [Venkatesh and Davis, 2000; Venkatesh et al., 2003]. Recent research has proposed that MIS theory needs to go beyond simply examining technology adoption [Jasperson et al., 2005; Burton-Jones and Straub, 2006]. These examinations of deep use posit that technology has levels of use that suggest longitudinal designs consistent with latent growth modeling [Burton-Jones and Straub, 2006]. Within an e-commerce context, trust studies have focused primarily on initial trust, which is trust when the user first makes contact with an e-vendor [McKnight et al., 2002; Serva et al., 2005a]. Early in the development of the trust nomological network (and in the early days of electronic commerce), this focus was understandable. Now that online vending has become commonplace and accepted, however, more research is needed to better understand how trust perceptions evolve as one s relationship becomes more developed with an e-vendor. The precursors and consequences of trust s trajectory need to be explored, as does trust s efficacy in predicting such distal outcomes as future buying behavior. Latent growth modeling also has the capability to improve MIS theory by opening new avenues of inquiry. Because time plays such a critical role in MIS phenomena [Pinsonneault and Kraemer, 1993], the nature of and precursors to many MIS constructs can be a critical factor in developing and refining MIS theory. In addition to computer anxiety and computer self-efficacy, some examples of constructs in the field of MIS that would benefit from longitudinal investigation include computer usage [Compeau et al., 1999], user acceptance of IT [Venkatesh et al., 2003], IT service quality [Watson et al., 1998], commitment to IT development [Newman and Sabherwal, 1996], user participation in systems development [Hunton and Beeler, 1997], changes in beliefs and attitude toward IT usage [Bhattacherjee and Premkumar, 2004], and the effect of computer technology training in the workplace [Venkatesh and Speier, 1999]. To the best of our knowledge, no one has used the LGM methodology to study change in these research areas. Latent growth modeling has other applications as well that can also improve understanding of MIS theory. LGMs can be used to test for factor invariance, confirming that construct validity is consistent over time. Factor invariance has largely been ignored in MIS research, but research has demonstrated that time can change the internal validity and meaning of MIS constructs. One study, for example, found that computer playfulness had changed since its original conceptualization [Serenko and Turel, 2007]. Such an application of the LGM approach could be used to confirm internal validity within a study, or perhaps determine if a construct has evolved in its meaning over time. Limitations of LGMs Despite the many advantages, LGM suffers from many of the limitations that are inherent in longitudinal research. MIS s reliance on cross-sectional research is understandable, at least to some extent: collecting longitudinal data requires extended time periods, exposes researchers to potential confounds, and necessitates increased buy-in from participating organizations. From a practical perspective, cross-sectional research can be completed more quickly and is often less demanding. Cross-sectional designs allow researchers to collect data within a relatively short time period, and researchers commonly assume that external confounds do not exist. 9 A longitudinal design may be most appropriate, therefore, when the expected effects are likely to occur within a relatively short period of time. This facilitates data collection, of course, but also reduces the chances that external factors will confound the results. When the proposed effects are likely to take months or even years (e.g., technology s effect on a company s organizational structure), the risk of external confounds may overly compromise a study s validity [Pinsonneault and Kraemer, 1993]. We note that the limitations of the extended time frame necessary for longitudinal data collection can be mediated to some extent through the use of a cohort sequential design [Duncan et al., 2006]. Longitudinal studies also suffer from issues related to missing data and attrition. For example, if a researcher s design requires data to be collected over four time points, some observations will likely be missing, simply because obtaining data from every sample unit at every occasion is often impossible. Attrition can also be a problem, as some subjects drop out before the study is completed. Advances in LGM allow for procedures that can account for some of these problems. LGM researchers advise against using the two commonly adopted procedures of dealing with missing data in practice listwise and pairwise deletion [Bollen and Curran, 2006]. Instead, they recommend using either direct maximum likelihood estimation or the multiple imputation method in dealing with missing data [Schafer and Graham, 2002]. Although a full discussion of the handling of missing values is beyond the scope of this study, articles have provided an excellent discussion of handling missing data within the context of SEM [Enders, 2006] as well as demonstrate fitting LGMs in the presence of missing data [Duncan and Duncan, 1994]. 9 This assumption is usually warranted, but confounds can occur during data collection for a cross-sectional study. Larger sample sizes, for example, often require multiple data collection periods. The occurrence of a significant event in the intervening periods (e.g., a company layoff, the death of a valued colleague, a corporate takeover) can still bias the results of a cross-sectional study. The extended time for longitudinal designs, of course, increase the chances that such a significant event will occur. 224

14 Memory effects could be an issue when subjects are simply checking boxes off on self-report survey forms that are administered repeatedly over time, as was the case in our study. This issue could be of concern if the time between successive administrations of the survey is short. Such effects can be mitigated somewhat by changing the order of items at each administration or by using parallel items that capture the same construct. Use of these alternatives will, of course, also increase survey design and data administration effort. The issue may not be a concern in situations where repeated measures data are objective. One example of objective data is repeated observations on data entry errors for personnel in charge of data entry in a data processing setting. Conclusion Motivated by the increased reliance on latent growth modeling outside of MIS as well as the lack of longitudinal analyses within MIS, this paper illustrates how LGM can provide additional insights into the nature of longitudinal phenomena. In promoting the use of LGMs, we also noted the similarities and differences with other more traditional longitudinal analysis approaches. Current research demonstrates, for example, that LGM has unique advantages compared to repeated measures ANOVA that MIS researchers need to incorporate where appropriate in their analyses. In particular, the flexibility of LGMs in allowing certain variables to be included simultaneously as dependent and independent variables in the same model, and the ability to allow for cross-domain analysis provides LGMs with an advantage over other approaches. We also noted some of the limitations inherent in LGMs, and noted the remedies recommended by researchers to overcome them. We highlighted some research areas within MIS where LGM applications appear to be most promising. As researchers, we believe that latent growth models greatly facilitate longitudinal analysis and provide great promise for improving our understanding of MIS theory. Other researchers have noted their advantages: LGM innovations are both exciting and too numerous to address [Hancock and Lawrence, 2006]. We encourage MIS researchers to explore the techniques outlined in this paper. Although longitudinal analysis presents challenges, the potential insights that are possible using latent growth modeling suggest the approach may open new possibilities in MIS research. REFERENCES Bhattacherjee, A. and G. Premkumar (2004) Understanding Changes in Belief and Attitude Toward Information Technology Usage: A Theoretical Model and Longitudinal Test, MIS Quarterly (28)2, pp Bollen, K. and P. Curran (2006) Latent Curve Models: A Structural Equation Perspective, Hoboken, NJ: John Wiley & Sons, Inc. Broos, A. (2005) Gender and Information and Communication Technologies (ICT) Anxiety: Male Self-Assurance and Female Hesitation, CyberPsychology & Behavior (8)1, pp Buche, M., L. Davis, and C. Vician (2007) A Longitudinal Investigation of the Effects of Computer Anxiety on Performance in a Computing-Intensive Environment, Journal of Information Systems Education (18)4, pp Burton-Jones, A. and D.W. Straub (2006) Reconceptualizing System Usage: An Approach and Empirical Test, Information Systems Research (17)3, pp Collopy, F., M. Adya, and J.S. Armstrong (1994) Principles for Examining Predictive Validity: The Case of Information Systems Spending Forecasts, Information Systems Research (5)2, pp Chan, D. and N. Schmitt (2000) Inter-Individual Differences in Intraindividual Changes in Proactivity During Organizational Entry: A Latent Growth Modeling Approach to Understanding Newcomer Adaptation, Journal of Applied Psychology (85)2, pp Compeau, D., C.A. Higgins, and S. Huff (1999) Social Cognitive Theory and Individual Reactions to Computing Technology: A Longitudinal Study, MIS Quarterly (23)2, pp Downey, J.P. and M. McMurtrey (2007) Introducing Task-Based General Computer Self-Efficacy: An Empirical Comparison of Three General Self-Efficacy Instruments, Interacting with Computers (19)3, pp Duncan, S., T. Duncan, and H. Hops (1996) Analysis of Longitudinal Data Within Accelerated Longitudinal Designs, Psychological Methods (1), pp Duncan, T., S. Duncan, and L. Strycker (2006) An Introduction to Latent Variable Growth Curve Modeling, New York, NY: Lawrence Erlbaum Associates. Duncan, T.E. and S.C. Duncan (1994) Modeling Incomplete Longitudinal Substance Use Data Using Latent Variable Growth Curve Methodology, Multivariate Behavioral Research (29)4, pp

15 Eisenberger, R., S. Hutchison, and D. Sowa (1986) Perceived Organizational Support, Journal of Applied Psychology (71)3, pp Enders, C.K. (2006) Analyzing Structural Equation Models with Missing Data in Hancock, G.R. and R.O. Mueller (eds.) A Second Course in Structural Equation Modeling, Greenwich, CT: Information Age, pp Gefen, D., D. Straub, and M. Boudreau (2000) Structural Equation Modeling and Regression: Guidelines for Research Practice, Communications of the Association for Information Systems (4), p. 35. Gurbaxani, V. and H. Mendelson (1990) An Integrated Model of Information Systems Spending Growth, Information Systems Research (1)1, pp Gurbaxani, V. and H. Mendelson (1994) Modeling vs. Forecasting: The Case of Information Systems Spending, Information Systems Research (5)2, pp Hamilton, J., P.E. Gagne, and G.R. Hancock (2003) The Effect of Sample Size on Latent Growth Models, American Educational Research Association, Chicago, IL. Hancock, G.R., W. Kuo, and F.R. Lawrence (2001) An Illustration of Second-Order Latent Growth Models, Structural Equation Modeling 8(3), pp Hancock, G.R. and F.R. Lawrence (2006) Using Latent Growth Models to Evaluate Longitudinal Change in Hancoick, G.R. and R.O Mueller (eds.) Structural Equation Modeling: A Second Course: Charlotte, NC: Information Age Publishing, pp Hunton, J.E. and J.D. Beeler (1997) Effects of User Participation in Systems Development: A Longitudinal Field Experiment, MIS Quarterly (21) 4, pp Marakas, G., M. Yi, and R.D. Johnson (1998) The Multilevel and Multifaceted Character of Computer Self-Efficacy: Toward Clarification of the Construct and an Integrative Framework for Research, Information Systems Research (9)2, pp McKnight, H., V. Choudhury, and C. Kacmar (2002) Developing and Validating Trust Measures for e-commerce: An Integrative Typology, Information Systems Research (13)3, pp Muthen, L. and B. Muthen (2002) How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power, Structural Equation Modeling (9)4, pp Newman, M., and R. Sabherwal (1996) Determinants of Commitment to Information Systems Development: A Longitudinal Investigation, MIS Quarterly (20)1. Pinsonneault, A. and K.L. Kraemer (1993) The Survey Research Strategy in Studies of Information Systems: Review and Critique, Journal of Management Information Systems (10)2, pp Schafer, J.L. and J.W. Graham (2002) Missing Data: Our View of the State of the Art, Psychological Methods (7), pp Serenko, A. and O. Turel (2007) Are MIS Research Instruments Stable? An Exploratory Reconsideration of the Computer Playfulness Scale, Information and Management (44)8, pp Serva, M., S. Benamati, and M. Fuller (2005a) Trustworthiness in B2C E-Commerce: An Examination of Alternative Models, DATA BASE for Advances in Information Systems (36)3, pp Serva, M., M. Fuller, and R. Mayer (2005b) The Reciprocal Nature of Trust: A Longitudinal Study of Interacting Teams, Journal of Organizational Behavior (26), pp Shah, R. and S.M. Goldstein (2006) Use of Structural Equation Modeling in Operations Management Research: Looking Back and Forward, Journal of Operations Management (24), pp Singer, J.D. and J.B. Willett (2003) Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence, New York, NY: Oxford University Press. Stoolmiller, M. (1994) Antisocial Behavior, Delinquent Peer Association and Unsupervised Wandering for Boys: Growth and Change from Childhood to Early Adolescence, Multivariate Behavioral Research (29), pp Thatcher, J.B. and P.L. Perrewé (2002) An Empirical Examination of Individual Traits as Antecedents to Computer Anxiety and Computer Self-Efficacy, MIS Quarterly (26)4, pp Venkatesh, V. and F.D. Davis (2000) A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Studies, Management Science (46)2, pp

16 Venkatesh, V. et al. (2003) User Acceptance of Information Technology: Toward a Unified View, MIS Quarterly (27)3, pp Venkatesh, V. and C. Speier (1999) Computer Technology Training in the Workplace: A Longitudinal Investigation of the Effect of Mood, Organizational Behavior and Human Decision Processes (79)1, pp Watson, R.T., L.F. Pitt, and C.B. Kavan (1998) Measuring Information Systems Service Quality: Lessons from Two Longitudinal Case Studies, MIS Quarterly (22)1. Willett, J.B. (1994) Measurement of Change in Husen, T. and T.N. Postlethwaite (eds.) The International Encyclopedia of Education, 2nd Edition, Oxford, UK: Pergamon Press, pp Willett, J. and A. Sayer (1994) Using Covariance Structure Analysis to Detect Correlates and Predictors of Individual Change Over Time, Psychological Bulletin (116)2, pp Wright, T. and D. Bonett (1993) Role of Employee Coping and Performance in Voluntary Employee Withdrawal: A Research Refinement and Elaboration, Journal of Management (19)1, pp APPENDIX A: LGM MODEL IDENTIFICATION Similar to all structural equation models, LGMs must achieve model identification that is, a model must constrain a sufficient number of parameters to result in positive degrees of freedom. Researchers have noted significant concerns with just-identified and under-identified models in research utilizing SEM, and recommend using overidentified models (i.e., where df > 0) whenever possible [Shah and Goldstein, 2006]. Determining the degrees of freedom in an LGM is similar to an SEM or causal measurement model. If p is the number of data waves and q is the number of parameters, the degrees of freedom for an unconditional LGM can be determined using the following expression: df = (1/2) * [p (p + 1)] q The above expression also can be used to determine the minimum number of data waves required for a hypothesized functional form. As an example, to fit a linear LGM, at least three waves of data are required, which is a necessary but not a sufficient condition to fit LGM [Bollen and Curran, 2006]. Figure 2 is useful in understanding the reason behind this. The LGM shown in Figure 2 is based on four waves of data (p = 4). These four waves provide a total of ten variances and covariances ((1/2) * [4 * (4 + 1)] = 10), plus four means, resulting in 14 observed pieces of information. Using these, the LGM model must estimate 9 pieces (q = 9) as follows: 4 error variances (one for each data wave), a variance and a mean for the slope and the intercept (4 pieces), and a covariance between the slope and the intercept. Thus, a linear model with four data waves will have (14 minus 9) 5 degrees of freedom. If all error variances were constrained to be equal, then our model would have 8 degrees of freedom (14 observed pieces minus 6 estimated pieces). With only two waves of data, we would have a total of five observed pieces of information ((1/2)*2*3 = 3 plus means for each wave = 5), which are insufficient to estimate 6 parameters even for the restricted model with equal error variances. APPENDIX B: STUDY QUESTIONS Computer Anxiety. Computer anxiety (CA) was measured using these four questions at each of the four time periods. A 1 5 rating scale was used, where lower average score indicated lower CA. I feel apprehensive about using computers. It scares me to think I could cause the computer to destroy a large amount of information by hitting the wrong key. I hesitate to use a computer for the fear of making mistakes that I cannot correct. Computers are somewhat intimidating to me. Instructor Supportiveness. Students were asked these two questions at each of the four time periods. Each instructor supportiveness indicator is the average of the two questions. The instructor cares about my general satisfaction with the course. The instructor takes pride in my accomplishments in the course. Computer Self-Efficacy. For all questions, students were first asked if they had the skill. If the answer was Yes, they then expressed their confidence that they could complete the task (10% 100%). The CSE indicator is the sum of the respondent s answers across the four areas (HTML, database, spreadsheet, and general computer skills), calculated at the end of the semester. 227

17 HTML Skills When creating a webpage in HTML...I believe I have the ability to create different sized headings....i believe I have the ability to create a hypertext link to another webpage....i believe I have the ability to add an image to a webpage....i believe I have the ability to use FTP to transfer my webpage to a Web server. I believe I have the ability to add a background image to my Web page. Database Skills Using a database package...i believe I have the ability to create a Select query....i believe I have the ability to create a summation query that aggregates business data....i believe I have the ability to create a query with a calculated field....i believe I have the ability to create a database table....i believe I have the ability to create a report. Spreadsheet Skills Using a spreadsheet package...i believe I have the ability to properly use relative and absolute addressing....i believe I have the ability to use the spreadsheet s built-in mathematical and statistical functions....i believe I have the ability to use the spreadsheet s logical functions (e.g., IF and VLOOKUP)....I believe I have the ability to create charts and graphs....i believe I have the ability to create a macro to automate a task....i believe I have the ability to create a pivot table to summarize business data. General Computer Skills Regarding computers in general I believe I have the ability to describe how a computer works. I believe I have the ability to install new software applications. I believe I have the ability to identify and correct common operational problems. I believe I have the ability to unpack and set up a new computer. I believe I have the ability to remove information that I no longer need from a computer. I believe I have the ability to use a computer to display or present information in a desired manner. APPENDIX C: SAMPLE STATISTICS CA = Computer Anxiety was calculated using items listed in Appendix A. T1 T4 indicate the measurement occasion. CSE = Computer Self-Efficacy; ISP = Instructor Supportiveness. Instructor supportiveness was a Likert scale question (see Appendix A). Gender indicates respondent was male (1) or female (0). Instructor was a dichotomous variable to indicate which instructor taught the respondent s section. Mean St. Dev. CSE T1-CA T2-CA T3-CA T4-CA T1-ISP T2-ISP T3- Instruc T4-ISP ISP -tor CSE T1-CA T2-CA T3-CA T4-CA T1-ISP T2-ISP T3-ISP T4-ISP INSTRUC TOR GENDER

18 APPENDIX D: MPLUS CODE FOR LGM MODELS USING Important note before running Mplus models: The code refers to studydata.txt file which can either be created or obtained by contacting the first or second author of the study. To create the file, enter the means, standard deviations and correlations given above (in that order). We have reported Mplus output for the CA Linear Model (below) in Appendix E for confirmation. The following MPlus models were used to generate the computer anxiety (CA) results included in the analysis. CA No-growth Model title: data: variable: model: output: CA No-Growth file is studydata.txt; type is correlations means stdeviations; nobservations are 230; names = cse anx1-anx4 is1-is4 instrucc gender; usevariables = anx1-anx4; i_anx anx1@1 anx2@1 anx3@1 anx4@1; standardized sampstat; CA Linear Model title: data: variable: model: output: CA Unconditional Linear file is studydata.txt; type is correlations means stdeviations; nobservations are 230; names = cse anx1-anx4 is1-is4 instrucc gender; usevariables = anx1-anx4; i_anx s_anx anx1@0 anx2@1 anx3@2 anx4@3; standardized sampstat; CA Freeform Model title: data: variable: model: output: CA Unconditional Free-form file is studydata.txt; type is correlations means stdeviations; nobservations are 230; names = cse anx1-anx4 is1-is4 instrucc gender; usevariables = anx1-anx4; i_anx s_anx anx1@0 anx2* anx3* anx4@1; standardized sampstat; CA Conditional on Gender and Instructor title: data: variable: model: output: CA Linear Conditional on Gender and Instructor file is studydata.txt; type is correlations means stdeviations; nobservations are 230; names = cse anx1-anx4 is1-is4 instrucc gender; usevariables = anx1-anx4 gender instrucc; i_anx s_anx anx1@0 anx2@1 anx3@2 anx4@3; i_anx s_anx on gender instrucc; standardized sampstat; CA Dual Growth Model with Distal Outcome title: data: variable: model: output: Dual growth model file is studydata.txt; type is correlations means stdeviations; nobservations are 230; names = cse anx1-anx4 is1-is4 instrucc gender; i_anx s_anx anx1@0 anx2@1 anx3@2 anx4@3; is_i is_s is1@0 is2@1 is3@2 is4@3; i_anx s_anx on is_i is_s gender instrucc; cse on i_anx s_anx is_i is_s; standardized sampstat; 229

19 APPENDIX E: MPLUS OUTPUT FOR THE LINEAR LGM 230

20 231

21 ABOUT THE AUTHORS Mark A. Serva is an assistant professor of MIS in the Department of Accounting and MIS, Alfred Lerner College of Business and Economics, University of Delaware. He has a Ph.D. from the University of Texas at Austin. His research interests are understanding trust and distrust, improving student learning, and latent growth modeling. He is also the associate director for the Institute for Transforming Undergraduate Education (ITUE), an organization that advocates for the use of student-centered pedagogies in higher education. Hemant V Kher is an associate professor in the Department of Business Administration, Alfred Lerner College of Business and Economics, University of Delaware. He has a Ph.D. in Production and Operations Management from the University of South Carolina. His research interests are in the area of modeling human learning effects in classroom and workplace settings. Jean-Philippe Laurenceau is an associate professor in the Department of Psychology, College of Arts and Sciences, University of Delaware. He completed a B.A. cum laude in Psychology from Cornell University and received his M.S. and Ph.D. from The Pennsylvania State University. His methodological interests include diary methods for studying interpersonal processes and applications of modern methods for the analysis of change. Copyright 2011 by the Association for Information Systems. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than the Association for Information Systems must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists requires prior specific permission and/or fee. Request permission to publish from: AIS Administrative Office, P.O. Box 2712 Atlanta, GA, , Attn: Reprints; or via from ais@aisnet.org. 232

22 . EDITOR-IN-CHIEF Ilze Zigurs University of Nebraska at Omaha ISSN: AIS SENIOR EDITORIAL BOARD Guy Fitzgerald Vice President Publications Brunel University Edward A. Stohr Editor-at-Large Stevens Institute of Technology CAIS ADVISORY BOARD Gordon Davis University of Minnesota Jay Nunamaker University of Arizona CAIS SENIOR EDITORS Steve Alter University of San Francisco CAIS EDITORIAL BOARD Monica Adya Marquette University Thomas Case Georgia Southern University Douglas Havelka Miami University Julie Kendall Rutgers University Ilze Zigurs Editor, CAIS University of Nebraska at Omaha Blake Ives Editor, Electronic Publications University of Houston Ken Kraemer University of California at Irvine Henk Sol University of Groningen Jane Fedorowicz Bentley University Michel Avital University of Amsterdam Evan Duggan University of the West Indies K.D. Joshi Washington State University Nancy Lankton Marshall University M. Lynne Markus Bentley University Ralph Sprague University of Hawaii Kalle Lyytinen Editor, JAIS Case Western Reserve University Paul Gray Founding Editor, CAIS Claremont Graduate University Jerry Luftman Stevens Institute of Technology Dinesh Batra Florida International University Mary Granger George Washington University Michel Kalika University of Paris Dauphine Claudia Loebbecke University of Cologne Richard Mason Southern Methodist University Hugh J. Watson University of Georgia Indranil Bose University of Hong Kong Åke Gronlund University of Umea Karlheinz Kautz Copenhagen Business School Paul Benjamin Lowry Brigham Young University Sal March Vanderbilt University Katia Passerini New Jersey Institute of Technology Mikko Siponen University of Oulu Rolf Wigand University of Arkansas, Little Rock DEPARTMENTS Information Systems and Healthcare Editor: Vance Wilson Don McCubbrey University of Denver Jan Recker Queensland University of Technology Thompson Teo National University of Singapore Fons Wijnhoven University of Twente ADMINISTRATIVE PERSONNEL James P. Tinsley AIS Executive Director Fred Niederman St. Louis University Jackie Rees Purdue University Chelley Vician University of St. Thomas Vance Wilson Worcester Polytechnic Institute Information Technology and Systems Editors: Sal March and Dinesh Batra Vipin Arora CAIS Managing Editor University of Nebraska at Omaha Sheri Hronek CAIS Publications Editor Hronek Associates, Inc. Shan Ling Pan National University of Singapore Raj Sharman State University of New York at Buffalo Padmal Vitharana Syracuse University Yajiong Xue East Carolina University Papers in French Editor: Michel Kalika Copyediting by S4Carlisle Publishing Services