METACOGNITION AND IT: THE INFLUENCE OF SELF-EFFICACY AND SELF-AWARENESS

Transcription

1 METACOGNITION AND IT: THE INFLUENCE OF SELF-EFFICACY AND SELF-AWARENESS Jane I. Gravill Ivey Business School The University of Western Ontario Deborah R. Compeau Ivey Business School The University of Western Ontario Barbara L. Marcolin Faculty of Management The University of Calgary Abstract Organizations are increasingly relying on employees to self-manage their learning needs. Therefore, it is important for individuals to accurately assess their IT knowledge because accurate self-assessment is critical to effective self-management. Metacognition represents individuals' self-monitoring and self-regulating abilities, and plays a key role in self-managed learning. This study examines two dimensions of metacognition - self-efficacy and self-awareness. We aim to understand how self-efficacy and self-awareness influence individuals metacognitive process and contribute toward increased effectiveness in self-managed learning. We argue that greater confidence in ability will result in increased self-awareness and learning outcomes in IT. Study findings suggest that increased computer self-efficacy is related to increased self-awareness and over-estimation. Low confidence in abilities was found to be related to under-estimation and lower levels of self-awareness. Therefore, under-estimation was found to be detrimental to learning outcomes while overconfidence was found to be beneficial. Further research is required to understand the threshold between beneficial and detrimental miscalibrated self-awareness. Keywords: End-user computing, user training, user characteristics Introduction Effective use of Information Technology (IT) in organizations requires ongoing user learning for employees to keep pace with the changing nature of IT and their roles. New learning requirements result from changes in both technology and tasks undertaken by individuals in the performance of their work. To meet these changing requirements, many organizations are increasing relying on employees to self-manage their learning needs through the use elearning, training portals or other self-directed computer training programs. Given the growing importance of self-managed learning, understanding metacognition (self-monitoring and self-regulation of abilities) is becoming an area of increasing research interest (e.g. Renner and Renner 2001; Zimmerman and Schunk 2001). As effective self-management begins with an individual s assessment of his or her capability (Mills 1983), it is important that users be able to accurately gauge their capabilities in the IT domain. Individuals need to be aware of the strengths and weaknesses of their abilities, to be in a position to effectively manage the development of their skills. However, recent research suggests that users have a tendency to miscalibrate and either over- or under-estimate their IT knowledge (e.g. Gravill et al. 2001; Marcolin et al. 2000). Over-estimation of knowledge and skills is a tendency which is common in many areas of human functioning, yet found to be detrimental to learning (Kruger and Dunning 1999). Hence, employees may not be effectively self-managing their learning process as organizations may believe Eighth Americas Conference on Information Systems 1055

2 Human-Computer Interaction Studies in MIS Metacognition is an individual s process of thinking about thinking, and is equivalent to self-regulation. If employees lack metacognitive skills and over-estimate their knowledge, they may not self-select themselves into the training programs they require. If employees under-estimate their knowledge, they will not fully utilizing or enhance the capabilities they do possess which decreases potential efficiencies for the organization. Thus, low levels of metacognition and misaligned self-assessments are clearly detrimental to effective self-managed learning and ultimately individual and organizational performance. Welldeveloped metacognition enhances an individual s performance by allowing them to optimize the capabilities they possess, and be aware of those that they do not. Self-efficacy, an individual s belief in their capability to perform a particular behavior, plays a key role in effective metacognition development, as does self-awareness of abilities. Zimmerman and Schunk 2001 build upon Zimmerman 1989 by describing the eight dimensions of metacognition (self-efficacy, self-awareness, resourcefulness, self-monitoring, goal setting, choice, self-motivation, attribution) through the lens of Social Cognitive Theory (Bandura, 1986), illustrating the importance of self-efficacy and self-awareness. This current study focuses on the relationship between self-efficacy and self-awareness in individuals' metacognitive process. We intend to determine how these two important factors influence individuals metacognitive process and contribute toward effective self-managed learning. To do so we draw upon the link Bandura has established between individuals CSE and action. Bandura explains that the central focus of self-efficacy theory is the dynamic interplay between self-referent thought, action and affect. Bandura has conducted studies linking self-efficacy and corresponding action supporting a causal relationship between the two (Bandura, 1986). Based upon this theoretical and empirical foundation, we suggest that individuals with greater CSE, more active in their day-to-day learning activities and increasing their interaction with their environment, will become more self-aware of their capabilities. An example may clarify this position. Consider an employee with low CSE who does not actively engage in daily activities at work such as conversations regarding topics like the release of new software or hardware features and what is currently available, how to discussions with others on the use of new or existing software features, or does not have the confidence to experiment with the variety of uses available for the software features resident on their desktop. This employee does not have as many interactions or opportunities to gain a better understanding of what they do, or do not, know. On the contrary, an employee with high CSE who is active in going about the process of their day-to-day work-life encounters more opportunities through these activities and their interaction with the environment, and receives more indication regarding the deficiencies and strengths of their abilities. High CSE employees will be active in interacting with others who have more or less knowledge than themselves, or with technology features that they have not previously seen, and through this enactment will develop a better sense of the strengths and weaknesses in their abilities. Hence, given Bandura s notion that CSE leads to action, we believe that individuals CSE will be related to their level of selfawareness regarding their abilities. Individuals with high CSE will have a clearer idea of what they do and do not know. This knowledge calibration is key to effective self-managed learning as it allows individuals to be aware of the skills they need to further develop through selected learning strategies, and those they are in a position to capably apply. Previous IS literature has studied the role of computer self-efficacy (CSE) in the training setting based on dependent variables such as learning strategy choice and learning outcomes (Compeau, Higgins and Huff 1999; Compeau and Higgins 1995a, 1995b; Compeau et al 1993) and other variables of interest (Webster and Martocchio 1992; Venkatesh and Davis 2000; Yi and Davis 2000), but has not addressed the role that CSE plays in determining individuals self-assessment of ability. CSE is defined as individuals' general beliefs about their abilities to competently use computers across multiple domains. The IS literature shows that CSE exerts a significant influence on individuals expectations of using computers, their motivation to participate in training (Gist et al. 1989; Hill et al 1987; Webster and Martocchio 1992; Yi and Davis 2001), their emotional reaction to computers, as well as their actual computer use (e.g. Compeau, Higgins and Huff 1999; Compeau and Higgins 1995a, 1995b; Venkatesh and Davis 2000). Other research on self-efficacy has indicated the relationship between self-efficacy and the adoption of technology products and innovations (Burkhardt and Brass; 1990; Hill et al. 1987). However, while much research has been conducted in the training context to understand the influence of CSE on individuals learning experience, we understand little regarding the important relationship between individuals CSE and self-awareness, and how the relationship between these two factors influence individuals metacognitive process and learning outcomes. This current study builds upon previous research in this area (Marcolin et al. 2000) and is part of a larger study (Gravill et al. 2001) designed to better understand the factors influencing accurate knowledge self-assessment. In order to investigate this research question regarding the relationship between CSE and selfawareness, this study examines individuals CSE, self-reported knowledge and demonstrated declarative and procedural knowledge. Declarative and procedural knowledge fall within the cognitive dimension of the User Competence Cube (Marcolin et al. 2000). Declarative knowledge would be measured by a paper and pencil test, while procedural knowledge would be most typically measured by a hands-on test Eighth Americas Conference on Information Systems

3 Gravill et al./metacognition and IT Research Model and Hypotheses The research model for this study is presented in Figure 1. The model represents the relationship between the three knowledge constructs in this study self-assessed knowledge, declarative knowledge and procedural knowledge as well as the influence of CSE on this relationship. The self-assessed knowledge construct represents subjects perception-based knowledge of their abilities to complete tasks in the particular domain. The declarative and procedural knowledge constructs represent subjects demonstrated knowledge as these constructs are based upon subjects knowledge test scores. The overlap between self-assessed and the demonstrated knowledge constructs is illustrated in this model by the shaded area where the construct boundaries overlap, representing accurate self-assessment of knowledge, or self-awareness from a metacognitive perspective. The larger the discrepancy between perceived and demonstrated knowledge constructs, the smaller the overlap and the less accuracy in selfjudgment, and the lower the individual's level of self-awareness. The fit model is used as it illustrates the similarities, as well as the regions of dissimilarity in the construct domains (Reisman 1988). Self -Assessed Knowledge (Self -Report) Declarative Knowledge what (P&P) General CSE + Procedural Knowledge how (Hands -On) Accurate Assessment of Software Knowledge (Self-Awareness) Figure 1. This study examines two hypotheses regarding the relationship between individuals CSE and their self-awareness, or the accuracy of their self-assessed knowledge. The first hypothesis considers the relationship between CSE and individuals' self-awareness. The second hypothesis examines the relationship in more detail, considering the impact of CSE on over- or under-estimation of knowledge. Computer Self-Efficacy and Self-Awareness CSE has been shown to influence individuals motivation to participate in computer training, their performance in software training, and their adoption of technology products and innovation (e.g. Compeau and Higgins 1995a, 1995b; Compeau, Higgins and Huff 1999; Webster and Martocchio 1992, Yi and Davis 2001; Burkhardt and Brass 1990). Participation in training has been shown to increase individuals metacognitive abilities (e.g. Renner and Renner 2001; Zimmerman 1985, 1986a, 1986b; Zimmerman and Schunk 1998, 2001). Effective training and learning experiences provide individuals with the opportunity to explore the boundaries of what they do and do not know, thereby increasing their ability to distinguish accuracy from error in their abilities and related judgments, hence exercising their metacognitive abilities and improving self-regulation. Therefore, due primarily to the behavioral outcomes (participation in training, exploration of boundaries of knowledge, etc.) driven by CSE, we propose that subjects with higher CSE will demonstrate greater accuracy in their knowledge self-assessments. Formally, 2002 Eighth Americas Conference on Information Systems 1057

4 Human-Computer Interaction Studies in MIS H1: Subjects with greater CSE will demonstrate greater accuracy in their knowledge self-assessments than will those subjects with less CSE. Computer Self-Efficacy and Over- Under-Estimation Individuals with stronger computer self-efficacy beliefs generally perceive themselves as able to accomplish more difficult tasks and have greater confidence regarding their ability to successfully perform computer-related behaviors (e.g., Compeau and Higgins 1995a, 1995b). Hence, individuals with higher CSE will be more likely to self-report higher levels of domain-specific knowledge, as these individuals believe more strongly in their abilities. Due to their increased confidence, we propose that, although individuals with higher levels of CSE will be those who are more accurate in their knowledge self-assessments, they will also be more likely to demonstrate over-estimation (rather than under-estimation) of their abilities. Formally, H2: Subjects with greater CSE will demonstrate over-estimation of software knowledge. Subjects with lower CSE will demonstrate under-estimation of software knowledge. Methodology Participants completed a questionnaire including demographic, other background information and self-reported knowledge. They also conducted a declarative and hands-on procedural knowledge test in a controlled environment. This research design allows us to examine the relationship between individuals' self-reported, declarative and procedural knowledge, and also allows for analysis of the influence of CSE on the level of accurate self-assessment (accurate self-assessment being self-reported knowledge equal to demonstrated spreadsheet software knowledge). A pilot study was initially conducted within a medium-sized organization. The data obtained in this pilot were primarily used to determine improvements required in the study design. Power calculations were conducted based upon this data to determine the appropriate sample size for our study. Subjects Subjects were volunteers from four large organizations in a medium-sized metropolitan area in the financial, retail, consulting, and distribution sectors. Subjects were screened to ensure that they used Microsoft Excel in performing their work, as Excel was the software domain used in the empirical portion of the study. All four organizations had standardized on Excel as the spreadsheet software package of choice for use within their operations. Notices were posted on the organizations intranets indicating that researchers were looking for volunteers to participate in a study designed to further understand the effectiveness of the self-managed learning process. To motivate volunteers of all abilities and avoid the problem of only high-achievers volunteering for participation, the intranet notice also included indication that we were looking for a group of volunteers with a wide range of expertise. We succeeded in obtaining a study sample comprised of subjects that demonstrated a wide distribution of knowledge. Measures Dependent Variables Self-reported knowledge was measured by six items that asked subjects to indicate, on a 7-point scale, their familiarity with different dimensions of spreadsheet functionality (editing, graphing, formulas, macro functions, database functions, printing functions). These specific items were chosen based on a review of the main categories of functionality in Excel (Winter 1999). Scores for this measure, and for the other knowledge measures, were converted to percentages for comparability purposes. The declarative knowledge measure consisted of 30 items, focusing on the same range of knowledge as the specific self-reported measure for comparability purposes. This measure was a pencil-and-paper multiple-choice test. Pre-testing was conducted on this Eighth Americas Conference on Information Systems

5 Gravill et al./metacognition and IT declarative knowledge measure and two weak items were revised. 1 The declarative knowledge test question topics were chosen randomly from a list of tasks available for inclusion on the Microsoft Office User Specialist (MOUS) certification examinations ( Questions based on these randomly selected topics were developed for the study. The MOUS examination tests the full range of skills required to be certified a specialist in using the MS Excel software. A random table generator was used to build a list of 30 random numbers, then the tasks associated with these numbers were selected from the MOUS examination task list for the declarative knowledge test questions. The procedural knowledge measure consisted of 22 items provided by an authorized Microsoft Excel MOUS certification test covering the same range of topics. The 22-item scale tested subjects knowledge of the 6 major categories of Excel software (Winter 1999) from various perspectives to produce an overall procedural knowledge test score for each subject. The test is divided into two sections, representing two levels of proficiency. The first level, called proficient in the Office 97 certification and core in later certifications, covers creating workbooks, modifying workbooks, printing workbooks, formatting worksheets, creating and applying ranges, using functions, using draw, using charts, and saving spreadsheets as HTML files. The second, or expert, level covers formatting worksheets, using lists, printing workbooks, auditing a worksheet, using advanced functionality, using macros, importing and exporting data, using templates, using multiple workbooks, and using workgroup functions (Microsoft 2002). Independent Variable Computer self-efficacy is described as individuals belief in their capability to perform computer-related behavior. This construct is measured using an 8-item scale that has been constructed, validated, and well-tested in the IS literature (e.g. Compeau and Higgins 1995a, 1995b; Compeau, Higgins and Huff 1999). Control Factor Social Desirability Bias Subjects also completed the Balanced Inventory for Desirable Responding (BIDR) measure (Paulhus 1988). This measure was included in our study to control for any social desirability bias that may be present in subjects self-reported knowledge assessments. Social desirability bias indicates whether subjects have a tendency to misrepresent themselves and respond in a positively biased manner. As evaluating knowledge can be sensitive, the possibility of social desirability bias (leading to either over- or under-estimation) had to be ruled out. The absence of social desirability bias is confirmed when the Social Desirability Bias scale is uncorrelated with the target variable (Fisher 2000), in this case the knowledge self-report construct. The BIDR measure consists of 40 items in total and is comprised of two subscales. The first subscale (20 items) reflects impression management, the tendency for subjects to deliberately misrepresent themselves to an audience. The second subscale (also 20 items) measures self-deception, which indicates whether subjects have the tendency to give self-reports that are honest but positively biased. Results Subjects Seventy-four subjects participated in the study, however only 67 fully completed both sections (proficient and expert) of the procedural knowledge test. Therefore, our study results are based upon the 67 subjects who had completed all of the measures. The 67 subjects were, on average, 37 years of age, and had 15 years of work experience. Seventy-one percent were women. They represented a variety of work roles, including managers (18%), professionals (24%), technical (23%) and other occupational groups. All had at least some university or college education. 1 The two questions that were replaced were specific to Excel 2000, and as it was our intention to include functionality resident in both Excel 97 and 2000, new questions were constructed Eighth Americas Conference on Information Systems 1059

6 Human-Computer Interaction Studies in MIS Table 1. Sample Demographics and Statistics Range % Mean Work Experience 1-35 yrs. 15 yrs. Age yrs 37 yrs Education Some Undergraduate Undergraduate Some Graduate Graduate 28.4% 47.3% 4.1% 20.2% Work Roles Managerial Professional Technical Admin, clerical, retail, operational Gender Women Men 18.0% 24.0% 23.0% 35.0% 63.0% 37% Full-Time Employees 95.0% Part-Time Employees 5.0% Table 2 shows the means and standard deviations for each of our study constructs, coupled with their reliability coefficients (Cronbach s alpha). All of the constructs showed acceptable reliability (Cronbach s alpha > 0.70), with the exception of the selfdeception component of Social Desirability Bias. The reliability observed in this study is lower than that typically observed for this measure (Paulhus 1988), but still reasonable. Given that this is a previously validated scale, with clearly established properties, no changes to the measure were made to attempt to improve reliability. Table 2. Descriptive Statistics and Reliability for Study Constructs Construct # Items Mean Std. Dev. C. Alpha Computer Self-Efficacy Self-Reported Knowledge Measure % Declarative Knowledge Measure % Procedural Knowledge Measure % Social Desirability Impression Management (possible range: 0 7) Social Desirability Self-Deception (possible range: 0 7) Table 3. Correlation Matrix for CSE and Knowledge Constructs CSE Self-Reported Knowledge Declarative Knowledge Procedural Knowledge CSE 1 Self-Reported Knowledge.479** 1 Declarative Knowledge ** 1 Procedural Knowledge.353**.755**.656** 1 **p < 0.01, n = 67 Relationship Between CSE and Knowledge Constructs Table 3 presents the relationships between CSE and the three knowledge constructs in this study. Though self-reported and demonstrated knowledge measures were significantly correlated as would be expected, these constructs demonstrated low shared Eighth Americas Conference on Information Systems

7 Gravill et al./metacognition and IT variance. Further, the relationship between CSE and the knowledge measures is lower still, and is significant only for selfreported and procedural knowledge. The correlation results indicate that the perceptual measures (self-reported knowledge and CSE) were more closely aligned with procedural rather than declarative knowledge. Thus, it appears that individuals reflected more on their procedural knowledge when making both general and specific judgments regarding their own capabilities. Interestingly, the test of correlations shows that there is some degree of overlap between individuals' CSE, self-reported and demonstrated software knowledge, but that the overlap is not high. Hypothesis Tests Computer Self-Efficacy and Self-Awareness Hypothesis one examines the influence of CSE on the relationship between individuals self-reported knowledge and their demonstrated declarative and procedural knowledge, proposing that subjects with greater CSE would demonstrate greater accuracy in their self-assessments of spreadsheet software knowledge. To test this hypothesis we examined the mean scores (as percentages) of each type of knowledge for subjects with low and high CSE. The logic for this approach is that if individuals are accurate in their knowledge self-assessments, their mean self-assessment score should be roughly the same as their mean score on the demonstrated knowledge tests. That is, if they indicate complete knowledge of the software (7 on the scale) they would be expected to score at or near 100% on the knowledge test. We conducted a repeated measures analysis of variance, using CSE as a between subjects factor. This produces three effects: a main effect of type of knowledge, a main effect of CSE and an interaction effect. A top and bottom quartile split to create the low versus high CSE groups was conducted as per previous research in this area (Kruger and Dunning 1999). The main effect of type of knowledge shows whether the three measures of knowledge are aligned. The main effect of CSE shows whether, overall, subjects with higher CSE score differently from those with lower CSE. The interaction effect is the result of particular interest. It shows whether the differences in the knowledge scores are the same or different for different levels of CSE. Thus, if the interaction is significant, we conclude that CSE influences individual s knowledge self-assessment accuracy. Post hoc tests are used to see where the differences are most pronounced. Table 4 shows the results of this analysis. The main effect of alignment is significant, indicating that self-reported, declarative knowledge and procedural knowledge are not aligned. The main effect of CSE is significant, indicating that those subjects with more CSE score differently on the three knowledge tests than subjects with less CSE. 2 Finally, the interaction effect is significant, suggesting that CSE influences the degree of alignment between self-reported, declarative knowledge and procedural knowledge. Hypothesis 1 proposes that the High CSE Group will be more accurate in their self-assessments of software knowledge. Drawing attention first to the relationship between self-reported and procedural knowledge scores in Table 4, results showed that the mean score for the self-report of the Low CSE group was significantly different than the mean score for that group's demonstrated procedural knowledge score. The self-report mean of did not lie within the upper or lower boundaries of the procedural knowledge scores that ranged from to 62.28, and hence was deemed significantly lower than procedural knowledge for that group. In reviewing the knowledge self-report scores for the High CSE group, it was evident that this groups' self-report mean of was so closely aligned with the demonstrated procedural knowledge score of that it was not significantly different. The mean self-report of did lie within the upper and lower boundaries of the procedural knowledge scores that ranged from to Therefore, based on the significant main and interaction effects demonstrated in this analysis and the Post Hoc analysis of the means conducted, we concluded that the High CSE group was more accurate than the Low CSE group in their knowledge self-assessments, supporting hypothesis 1. Examination of the means for self-reported knowledge compared to subjects' demonstrated declarative knowledge did not support hypothesis 1. For the low CSE group, self-reported knowledge was not significantly different from declarative knowledge, while for the high experience group, self-reported knowledge was significantly higher than declarative knowledge. Therefore, hypothesis 1 is partially supported. The less consistent nature of the declarative knowledge findings may be due to results of previous research on the knowledge acquisition process, indicating that as knowledge advances from the declarative to procedural stages, initial declarative knowledge may be lost (Anderson and Finchman 1994; Anderson 1993). 2 Post hoc tests show that the differences are in the self-report measure rather than the declarative or procedural knowledge measures. Subjects with low vs. high self-efficacy do not score significantly differently on either declarative or procedural knowledge Eighth Americas Conference on Information Systems 1061

8 Human-Computer Interaction Studies in MIS Table 4. CSE influences degree of alignmnet SS DF F Sig Alignment Main Effect * CSE Main Effect * Alignment C CSE * Error-Within Error-Between Post Hoc Comparison of Means 95% Confidence Interval Low CSE Group Self-reported Knowledge Declarative Knowledge Procedural Knowledge High CSE Group Self-reported Knowledge Declarative Knowledge Procedural Knowledge Mean Std. Error Lower Bound Upper Bound Computer Self-Efficacy and Over- Under-Estimation Hypothesis 2 considered the relationship between individuals CSE and the direction of their self-assessment accuracy (over- or under-estimation). Table 4 indicates that the High CSE group over-estimated both their declarative and procedural knowledge, though their procedural knowledge over-estimation is not significant in this respect. The Low CSE group under-estimated on both accounts, though only procedural knowledge is significant. Therefore, though only partially supported statistically, these findings clearly indicate the direction of the effect of CSE on the accuracy of knowledge self-assessment. Social Desirability Bias Regression results indicated that Social Desirability Bias results for image management and self-deception were not significantly related to subjects overall knowledge self-assessment scores (p<0.33 for self-deception overall, p<0.63 for image management overall). This non-significant overall finding validated the knowledge self-assessments and study results for our hypotheses in this respect by ruling out an alternative social desirability bias explanation for our results and confirmed that subjects were not misrepresenting their overall scores based on social desirability influences. We suggest that it is important to address Social Desirability Bias when relying on self-reported measures of knowledge to understand whether subjects are inflating their self-assessments for these reasons. Our subjects did not demonstrate significant social desirability bias influences in their self-assessments, but without ruling out the inflating influence of social desirability bias, researchers cannot be certain whether their results based on self-reported responses are valid or biased due to social influences. Discussion Our study findings indicate that subjects in the High CSE group were more accurate in their knowledge self-assessments based on their procedural knowledge. This finding is important toward further understanding of effective self-managed learning, as accurate self-assessment is the first step toward effective self-managed learning. These results tell us that individuals with more Eighth Americas Conference on Information Systems

9 Gravill et al./metacognition and IT confidence in their abilities with computers are likely to be more effective in self-manage their learning. This finding suggests that efforts toward increasing individuals' CSE, or their confidence in their ability to perform, may ultimately improve their selfregulation skills. This makes sense, as individuals with more confidence tend to participate more in activities that explore the boundaries of what they do and do not know, thereby increasing their ability to distinguish accuracy from error in their abilities and related judgments, exercising their metacognitive abilities and improving self-regulation. While we found that subjects in the High CSE group were more accurate in their knowledge self-assessments, we also found that this group tended toward over-estimation and individuals in the Low CSE group significantly under-estimated their knowledge. The over-estimation finding for the High CSE group may be attributed to their increased confidence in their abilities to use computers, causing them to tend toward exaggeration when estimating their domain-specific knowledge. Given that this group were also more self-aware of their knowledge, this study suggests that the over-estimation demonstrated by the High CSE group may have positive consequences and supports the notion that tendency toward over-estimation of knowledge may be a healthy and beneficial human condition (Alba and Hutchinson 2000). This finding suggests that perhaps a certain degree of overconfidence in abilities may not be detrimental to learning as previous research has suggested (Kruger and Dunning 1999). On the other hand, under-estimation and low confidence do seem to be related to lower levels of self-awareness. Further research is required to understand the threshold between beneficial and detrimental miscalibrated self-awareness to understand how much over-confidence is beneficial to self-managed learning and learning outcomes. Limitation of the Study No study is without limitations. For this study, two limitations in particular should be noted. First, the research model was only tested in one software domain, therefore generalizations to other domains cannot be assumed. Testing our model in other domains is important for better understanding and confirmation of our results, and is required for further generalization of our findings. Second, the subjects who participated in our research study were volunteers, and although we undertook precautions regarding this and were successful in obtaining a sample consisting of a wide distribution of knowledge, we cannot be certain the study sample is representative of all employees of the four organizations that participated in the study. Conclusions and Directions for Future Research This study provides a step toward further understanding the nature of the relationship between individuals' CSE, self-reported and demonstrated declarative and procedural knowledge. This study illustrated that increased confidence in ability can lead to improved self-managed learning. Individuals with higher CSE were more self-aware as they demonstrated greater accuracy in their knowledge self-assessments, though they did tend to over-estimate their domain-specific abilities. This study assists in developing a further understanding regarding self-efficacy and self-awareness as elements of individuals' metacognition, which can assist both researchers and practitioners in the process of better understanding and more effective facilitation of self-regulated self-managed learning. Given the findings that we have observed in the current study based upon the relationship between CSE, self-reported and demonstrated knowledge - we are motivated to conduct future research in this area to further understand how individuals' perception of their abilities are influenced, and in turn, influence the effectiveness of the self-managed learning process. Metacognition plays a key role in effective self-managed learning, and research to further understand how other dimensions of metacognition such as goal-setting, attribution, self-monitoring, resourcefulness, self-motivation and strategic choice can contribute toward effective learning in the IT context is recommended. References Alba, J. and Hutchinson, J Knowledge calibration: What Consumers Know and What They Think They Know, Journal of Consumer Research, (27:2), 2000, pp Anderson, J.R. Acquisition of Cognitive Skill Psychological Review, (89:4), pp , Anderson, J.R and Fincham,. Acquisition of Procedural Skills From Examples, Journal of Experimental Psychology: Learning, Memory and Cognition, (20),1994, pp Bandura, A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, Prentice-Hall, Eighth Americas Conference on Information Systems 1063

10 Human-Computer Interaction Studies in MIS Burkhardt, M. and Brass, D. "Changing Patterns or Patterns of Change: The Effects of a Change in Technology on Social Network Structure and Power", Administrative Science Quarterly, (35), 1990, pp Compeau, D., Olfman, L., Sein, M., & Webster, J. End User Training and Learning: Introduction to the Special Section, Communications of the ACM, (38:7), 1995, pp Compeau, D. and Higgins, C. "Computer Self-Efficacy: Development of a Measure and Initial test". MIS Quarterly, 23(2), 1995, pp Compeau, D. and Higgins, C. "Application of Social Cognitive theory to Training and Learning", Information Systems Research, 6(2), 1995, pp Compeau, D., Higgins, C. and Huff, S. " Social Cognitive Theory and Individual Reactions to Computing Technology: A Longitudinal Study. MIS Quarterly, 23(2), 1999, pp Crown, D.P., and Marlowe, D. A New Scale of Social Desirability Independent of Psychopathology, Journal of Consulting Psychology, (24:4), 1988, pp Fisher, R. The Future of Social-Desirability Bias Research in Marketing, Psychology & Marketing, (17:2), 2000, pp Gist, M. Schwoerer, C. and Rosen, B. "Effects of alternative training methods on self-efficacy and performance in computer software training", Journal of Applied Psychology, 74, 1989, pp Gravill, J., Compeau, D., and Marcolin, B. Frame-of-Reference Affects on the Accuracy of Self-Assessed User Competence, Proceedings of the International Conference for Information Systems, Hill, T., Smith, N. and Mann, M. "Role of Efficacy Expectations in Predicting the Decision to Use Advanced technologies: The Case of Computers", Journal of Applied Psychology, (72:2), 1987, pp Kruger, J. Lake Wobegon Be Gone! The "Below-Average Effect" and the Egocentric Nature of Comparative Ability Judgments, Journal of Personality and Social Psychology, (77:2), 1999, pp Kruger, J., and Dunning, D. Unskilled and Unaware of it: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-assessments, Journal of Personality and Social Psychology, (77:6), 1999, pp Marcolin, B., Compeau, D., Munro, M.C., & Huff, S. Assessing user competence: Conceptualization and measurement, Information Systems Research, (11:1), 2000, pp Microsoft. Microsoft Office User Specialist Certification Requirements, Accessed February 28, Mills, P.M. Self-Management: Its Control and Relationship to Other Organizational Properties, Academy of Management Review, (8), 1983, pp Munro, M., Huff, S., Marcolin, B., & Compeau, D. Understanding and Measuring User Competence, Information and Management, December 1997, pp Nunnally, J.C. Psychometric Theory, 2 nd ed. McGraw Hill, New York, Reisman, A. On Alternative Strategies for Doing Research in the Management and Social Sciences, IEEE Transactions on Engineering Management, (35:4), 1988, pp Renner, C. H. and Renner, M.J. But I Thought I Knew That: Using Confidence Estimation as a Debiasing Technique to Improve Classroom Performance Applied Cognitive Psychology, (15), 2001, pp Venkatesh, V. and Davis, F. A Model of the Antecedents of Perceived Ease of Use: Development and Test, Decision Sciences, (27:3), 1996, pp Venkatesh, V. and Davis, F. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies Management Science, (46:2), 2000, pp Yi, M.Y. and Davis, F. Improving Computer Training Effectiveness for Decision Technologies: Behavior Modeling and Retention Enhancement, Decision Sciences, (32:3), 2001, pp Webster, J. and Martocchio, J. "Microcomputer Playfulness: Development of a Measure With Workplace Implications" MIS Quarterly, June, 1993, pp Weick, K. The Social Psychology of Organizing (2 nd edition), Addison-Wesley, Reading, MA, Winter, R. Microsoft Office User Specialist Excel 2000, Macmillan, Indianapolis, Zimmerman, B. "A Social Cognitive View of Self-regulated Academic Learning" Journal of Educational Psychology, (81:3), 1989, pp Zimmerman, B. and Schunk, D., 'Self-Regulated Learning and Academic Thought', Lawrence Erlbaum Associates, Eighth Americas Conference on Information Systems