ROLE OF ANXIETY AND COMPUTER EFFICACY ON LEARNING IN SPREADSHEET BASED MULTIPLE- CHOICE AND CONSTRUCTED RESPONSE TESTS

Transcription

1 ROLE OF ANXIETY AND COMPUTER EFFICACY ON LEARNING IN SPREADSHEET BASED MULTIPLE- CHOICE AND CONSTRUCTED RESPONSE TESTS Anil Singh School of Business University of Texas at Brownsville 80 Fort Brown, Brownsville Texas, TX Phone: Fax: Dean: Dr. Rafael Otero Vikram Bhadauria College of Business Southern Arkansas University 100 E. University, Magnolia, AR Ph Fax: Dean: Dr Lisa Toms Anurag Jain Bertolon School of Business Salem State 352 Lafayette St. Salem, Ma Phone: Anil Gurung Marshall University Lewis College of Business One John Marshall Drive Huntington, WV Telephone:

2 ABSTRACT The objective of this study is to investigate antecedent factors to learning spreadsheets. A secondary objective is to investigate the best measurement of learning in spreadsheets. We explore differences in two measures of learning spreadsheets: multiple choice test and constructed response tests. To achieve our objective we develop a model to identify the antecedents to learning in Spreadsheets. The empirical investigation using PLS method indicates the following: 1. Computer self-efficacy is inversely related to anxiety. 2. Both computer self efficacy and anxiety positively influence learning on multiple choice as well as constructed response tests. 3. There appears to be a significant difference between the variances of multiple choice grades and constructed response grades and 4. The data analysis also revealed that female students tend to have to have higher anxiety levels than male students. INTRODUCTION Research in effective teaching pedagogy has looked for more efficient ways of achieving maximum learning and also looked for properly measuring the learning that takes place. In this paper we study the role of anxiety; computer efficacy and gender on learning in spreadsheet based multiple- choice and constructed response tests. Spreadsheets are a useful tool to collate and present business information in a compact and meaningful manner. Organizations use Spreadsheets extensively for business operations and decision making. Universities and business schools, therefore, have integrated Spreadsheets in their curriculum for business majors. Examining antecedent factors that influence a student s learning in a course on Spreadsheets is important for academicians and practitioners alike. The objective of this study is to investigate antecedent factors that can predict a student s academic success in a Spreadsheets course. Findings of this research can help instructors make their course design more effective. A secondary objective of this study is also to investigate if there are significant differences between a multiple choice questions based Test and a problem solving based Test. The rest of the paper is organized as follows. First, we do the review of extant literature, followed by the research model. Next, we discuss the research method and data collection, followed by data analysis. The discussion section follows next, followed by conclusions and limitations. Computer Anxiety LITERATURE REVIEW A dictionary definition of anxiety is given as distress or uneasiness of mind caused by fear of danger or misfortune. In literature, computer anxiety has been defined as the fear of computers when using it or when considering possible computer use (Heinssen et. Al 1987). Building upon the work of Raub (1981), Igbaria and Parsuraman (1989) described computer anxiety as the tendency of individuals to be uneasy, apprehensive, or fearful about current or future use of computers. The consequences of computer anxiety may include decreased use, and in some cases, even avoidance of information technology (Igbaria and Parsuraman 1989, Igbaria et al, 1994). For this study, computer anxiety is conceptualized as fear related to the use of spreadsheets. Students may be apprehensive of using spreadsheets for fear of making mistakes that they may not be able to correct.

3 Computer self efficacy The concept of Computer self-efficacy was formulated in information systems research, which delved into the behavior of individuals (i.e. self-efficacy) in the context of the computing technology (Campeau and Higgins, 1995). Computer self-efficacy has been defined as, judgment of one s capability to use a computer (Campeau and Higgins, 1995). Self-efficacy is derived from social cognitive theory (Bandura, 1986). According to this theory, self-efficacy alludes to the judgment that individuals make of their ability to do a task. It is the expectation that an individual forms, that can govern his or her learning in tasks. Therefore, the higher the self-efficacy an individual has, the greater should be the effort exerted by that individual (Bandura and Cervone, 1983, 1986). Learning Learning was operationalized through two variables: Multiple Choice Tests (MCT and Constructed Response Test (CRT). Both of these are discussed below. Multiple Choice Test (MCT): The multiple choice test typically involved questions with many answer choices among which only one is the right answer. The topics covered functions and their arguments such as PV, PMT, FV, SUMIF, COUNTIF etc. The students were shown various solutions to a particular question. They had to choose the correct one. Additionally there were some questions related to questions too. Constructed Response Test (CRT): The constructed response test was a hands on approach as compared with the multiple choice Test. In the constructed response test, the students had to download a spreadsheet file, work on it to complete a set of tasks. The spreadsheet functions they were to use were: VLOOKUP, HLOOKUP, COUNT, COUNTIF, SUM, SUMIF, IF, NESTED- IF in conjunction with AND and OR. To complete the tasks and use these function in excel successfully also required knowledge of referencing from another cell, worksheet and file. Knowledge of relative as well as absolute referencing was also needed. In contrast to multiple choice questions, constructed response tests require a more hands on approach. Control variables To control for externalities, we referred to related studies and controlled for experience, Age and Gender (Martocchio and Webster1992). These were either used as controlled variables or variables of interest. Role of Anxiety in Learning HYPOTHESIS DEVELOPMENT Results from Past studies are mixed regarding the relationship between computer anxiety and learning. While some studies show little support for the relationship (Desai and Richards 1998, Kernan and Howard 1990), there are studies that suggest that a negative relationship exists (e.g. Keeler and Anson 1995). As suggested by Bandura s theory, computer anxiety is a cognitive variable that influences a person s behavior of computer use (Bandura 1986). In light of these

4 findings, it is important to investigate the relationship of computer anxiety with learning outcomes. Thus: H1: Anxiety negatively influences learning on multiple choice tests H2: Anxiety negatively influences learning on constructed response tests Role of Self-Efficacy in reducing Anxiety Although computer anxiety may lead to decreased use of computer or some sort of decrease in learning there are other factors that could possibly lower the impact of computer anxiety. Having a positive attitude towards information technology can contribute to the use. In the context of our study, students with computer anxiety may be able to avert the decrease in learning by building a positive attitude towards using spreadsheets. Hence, we posit that H3: Self-efficacy negatively influences anxiety Role of Self-Efficacy in learning in Multiple Choice tests and in Constructed Response tests Personal factors play a role in use computer related tasks and learning. Lewis et. al. (2003), point to self-efficacy as being one of the significant factors. Others (e.g. Akbulut and Looney, 2007) empirically suggest there is a strong association between self efficacy and the students wanting to take up computers as a discipline. Simmering and Posey (2009) empirically validate the association between self-efficacy and online class learning success. Mykytyn et al., (2008) through the use of problem based learning pedagogy suggest that higher levels of computer selfefficacy can provide higher levels of positive outcomes for students in computer application classes that emphasize spreadsheets. Therefore Computer-self efficacy becomes an important antecedent to learning in computer related activities such as Spreadsheets. Hence, we posit that: H4: Self-efficacy positively influences learning on multiple choice tests H5: Self-efficacy positively influences learning on constructed response tests Differences between Multiple Choice responses and Constructed Response tests One criticism against Multiple-choice tests is that they do not effectively measure learning (Chan & Kennedy 2002). Multiple-choice may measure only specific aspects like recall and therefore a better evidence of learning is when the students are able to perform what they are asked to do by assimilation of what they have learnt in the form of constructed response tests. Just multiple choice questions may fall short of measuring true learning as the students may be expected to just read the book and answer from among choices. Thus: H6: Constructed response tests are better measurements of learning than multiple response tests. RESEARCH MODEL

5 Anxiety H1(-) Learning MCT (L1) Experience H3(-) Gender Self-Efficacy H4(+) H5(+) H2(-) Learning CRT (L2) Age Figure 1: Researech Model RESEARCH METHOD AND DATA Data Source The subjects were undergraduate business students taking the first course of Information Systems. This course included Spreadsheets. The student samples were taken from four information systems sections of the same course taught by the same instructor across two years. In case of multiple choice questions, the average of two tests related to spreadsheets was taken as proxy to learning. In case only one test was taken, the score of that test was treated as the proxy for learning. When both tests were not taken, the student was dropped from the survey. In case of constructed response test, one test was given. If the student missed this test, the student was dropped from the survey. The survey responses combined with their learning completed the data set. Sample Characteristics 55 % of the respondents were females. Mean of age was 24. Average of learning in Multiple Choice test was 6.2 (out of 10), Average of learning in Constructed Response test was 7 ( out of 10) and mean of past experience was Methodology The data was analyzed using the Partial Least Square (PLS) technique (Software: PLS Graph Ver. 3 Courtesy Dr. Wynne W. Chin, University of Houston). PLS, is a is a second generation data analysis method that not only assesses the structural model but also assesses the measurement model thereby enabling simultaneous analysis of the factors along with the testing of the hypothesized relationships between and among measures (Marcoulides and Saunders 2006). PLS allows the mapping of both formative as well as reflective measures.

6 While there are cases where PLS has been considered a good technique for small sample size, others suggest practicing caution with small sample size (Marcoulides and Saunders 2006) and have suggested a ratio of 15 respondents to one parameter to minimize problems with deviations from normality. While our pilot test consists of 44 data points; we aim to get additional 150 more data points by the end of May. A total sample of 194 will sufficiently exceed the suggested requirement of 60 (4 x 15) responses to satisfy the above criteria. AVE and Latent Variable Correlations are shown in Table 1. In the secondary study, we test for difference between constructed response test and the multiple choice test, differences in the tests as well as anxiety and self-efficacy among the genders. Experience Anxiety -0.19* Learning MCT (L1) -0.27*** Gender Self-Efficacy 0.20** -0.33*** 0.13* Learning CRT (L2) Age Figure 2: Measurement Model Table 1: AVE and Latent Variable Correlation Anxiety Self- Efficacy P1(MCT) P2(CRT) Age Experience Gender AVE Anxiety Self- Efficacy P1(MCT) P2 (CRT) Age Exp. Gender Testing for Common Method Variance

7 To check for common method variance, we used the PLS approach suggested by Podsakoff, MacKenzie, Lee, & Podsakoff (2003). The items loaded more on their own constructs than on the common construct. This confirmed the unlikely chance of common method bias. Primary Test After including in control variables Gender, age, past experience in the PLS model we tested the hypotheses presented in the previous section (Table 2). Hypothesis H1 was rejected at the P< 0.01 level. There was evidence of anxiety negatively effecting MCT (P1). Hypothesis H2 was rejected at P< 0.01 level revealing evidence of anxiety negatively effecting CRT (P2). In Hypothesis H3, we found evidence of self-efficacy decreasing anxiety at P< 0.01 level. Hypothesis H4 was rejected at the P< 0.01 level showing evidence of self-efficacy influencing MCT (P1). Hypothesis H5 regarding self-efficacy influencing CRT(P2) was rejected at the P< 0.01 level. Table 2: Table of Hypothesis Mean Standard Error t Statistic Anxiety -> MCT(P1)-multiple choice (H1) Anxiety ->CRT(P2) constructed response (H2) Self-Efficacy -> Anxiety (H3) Self-Efficacy -> MCT(P1)- multiple choice (H4) Self-Efficacy -> CRT(P2)- constructed response (H5) Test for differences among variances of MCT and CRT scores ( F test ) (H6) F stat: F: critical 1.66 Results Reject at p < 0.1 Secondary Test We performed t tests (table 3) to test for differences between Learning type1 (MCT) and Learning type 2 (CRT). Although grades in multiple choice tests were significantly better than grades in constructed response tests, the variance in constructed response tests were 7.38 compared to 1.45 in multiple choice tests. We used F to test for differences among variances of the two measurements and found significant difference between the two. The variance is higher among CRT scores as compared to MCT scores (figure 3: Box Plot). Table 3: T-test between the tests CRT MCT Mean

8 Variance Observations Pooled Variance Hypothesized Mean Difference 0 Df 86 t Stat P(T<=t) one-tail t Critical one-tail P(T<=t) two-tail t Critical two-tail From the figure we can observe that the relationship between anxiety and constructed response tests is more than between anxiety and the multiple choice tests. Gender showed a negative effect (females: 1 males: 0) in both the tests although this was not significant at even at p < 0.l. This can also be seen at the latent variable correlations tables (pair wise correlation gender MC = and gender to CR tests T test to test for gender differences in self-efficacy showed negative but not (t-stat = -0.80) significant differences. T test to test for gender differences in anxiety showed negative and significant (t-stat = 1.63) differences at the p< 0.1. Box-and-Whisker Plot CRT MCT Discussion and Conclusion Figure 3 Box plot This paper answers two questions: 1.What antecedent influences learning in spreadsheets? 2: What is the best measurement of learning? Using data collected from students, we developed a model to identify influential antecedents to their learning in the tests on Spreadsheets. We found differences among learning in multiple choice tests and constructed response tests in

9 spreadsheets. Prior literature suggests the role of anxiety and computer self-efficacy as the two most influential factors in the academic success of students. PLS method used on the data revealed the following evidence: 1. Computer self-efficacy has a significant negative influence on anxiety. 2. Self efficacy has a significant positive influence on both measurement of learning: multiple choice as well as constructed response tests. 3. Anxiety negatively influences learning on both multiple choice as well as constructed response tests. 4. There was a significant difference in the variance of multiple choice grades and constructed response grades and 5. The only significant difference among genders in all constructs was in case of anxiety where female students were found to have higher anxiety. The determinant factors of academic learning have always been an area of interest in pedagogical research. From the latent variable scores we can see that both anxiety and self-efficacy are important antecedents to a student s academic learning. Despite the direct influence of selfefficacy on both learning indicators, anxiety still plays a major role in reducing both the learning measures. Instructors need to find ways and means to reduce anxiety and improve computer selfefficacy. Related research in this area has identified playfulness as one antecedent which reduces anxiety and increases self efficacy. Using conclusions from the study, instructors could challenge the students in doing and repeating day to day fun activities related to Spreadsheet which could indirectly influence learning. (References are available upon request from Anil Singh)