Beth J. Massie. The Effect of Training on User Buy-In. A Master s Paper for the M.S. in I.S degree. April, 2009. 13 pages. Advisor: Dr. Robert Losee. This study describes the results of a questionnaire survey of members in the Communication and Knowledge Services teams of IntraHealth International s Chapel Hill Headquarters in February 2009. The survey was conducted to determine the relationship between training, user self-confidence in his/her ability to use a technology, the emotional response to using that technology, and the actual usage of that technology. The training did improve the emotional response of some employees to the use of the blog as well as the confidence in the employee in the employee s ability to use the blog. There was no change in the usage of the blog: the only employee who posted after the training had also posted once before the training. While the results of this study were not as astounding as the researcher could have hoped, the researcher believes that if this study were done in the future, several changes would result in improved results. Headings: Training Computer software In-service education
THE EFFECT OF TRAINING ON USER BUY-IN by Beth J. Massie A Master s paper submitted to the faculty of the School of Information and Library Science of the University of North Carolina at Chapel Hill in partial fulfillment of the requirements for the degree of Master of Science in Information Science. Chapel Hill, North Carolina April 2009 Approved by Dr. Robert Losee
1 Table of Contents Introduction Literature Review Methodology Analysis Conclusions
2 Introduction Although the business world agrees that having buy-in for new corporate wide technology assists in the adoption of said technology, few people agree on the best way to create buy-in. Buy-in is a combination of self-confidence in one s ability to use the technology in question and one s emotional response towards the technology while using it. The combination of one s rating of one s ability to use a technology and the often corresponding emotional response to actually using it influence the frequency and extent of use of the technology. This study examined the usage of IntraHealth International s internal SharePoint all staff news blog by members of the Communications and Knowledge Services Teams. The intention of this study was to examine the relationship between training, user selfconfidence in his/her ability to use a technology, the emotional response to using that technology, and the actual usage of that technology. The results of this study were rather unrevealing: although there was a slight improvement in the emotional response and selfconfidence of several of the users, there was no change in the usage of the blog. However, the researcher suspects that these results could be improved if the study were redone with several significant changes. Literature Review While training is often advocated as a useful tool to increase usage of a technology new to an individual, 1 a review of the available literature revealed few studies
3 examining the relationship between training, buy-in, and usage of a technology. However, the individual aspects of the training, self-confidence, and emotional response in relation to software adoption have been studied. A study by K Nowak and M Krcmar showed that a relationship between a person s belief in their capability to effectively use a computer and their likelihood to work in a computer related field exists. 2 Additionally, they examined the effect of forced usage of a computer on the individual s self-efficacy rating. Rather than being an actual skill they possess, self-efficacy is the rating of the belief of the skills one believes one possesses. Nowak and Krcmar concluded that by forcing an individual to work with a computer, one can influence an individual s self-efficacy rating. 3 That same year, K Hartzel looked at the effect self-efficacy had on software adoption and concluded that software adoption occurs when the user s level of self-efficacy is manipulated as the user needs to believe he or she has the ability to use the software successfully. 4 Training incorporates the forced use of a specific software and therefore should increase the user s self-efficacy rating. Although an individual can be trained into believing they can successfully use a given program, the question arises regarding their emotional response to using said program. For example, an individual may believe that they are perfectly capable of successfully using a piece of software but never extensively use it due to dislike of how the software works and frustration arising from using it. Thus, it seems obvious that one s emotions towards technology also affect software adoption. Although this idea has not been extensively studied, in 2007, Kay and Loverock decided to examine the relationship between the four emotions anger, anxiety, happiness, and sadness and
4 learning new software. 5 Their study revealed that long term training programs, while successfully decreasing feelings of anxiety and anger, had no effect on happiness or sadness. 6 Part of the lowering of anxiety and fear may be attributed to a reduction of the fear of loss of status. As Co, Patuwo, and Hu point out, When a new technology is introduced, those who use it or own it see their power and status enhanced, while those who do not use it see the reverse. 7 By training all employees on the new software, the aspect of fear should be eliminated as all employees would have the ability to use the new software and thus, all would own it. Additionally, the familiarity achieved through training should positively impact the opinion of the employees towards the software. A study conducted by Curran and Meuter on the adoption of self-service technology revealed that the opinion of customers regarding self-service banking technologies affected which technologies were adopted. 8 That is, those self-service technologies which were thought highly of were more likely to have been adopted while the remaining self-service technologies (those that were not highly thought of) were less likely to have been adopted. 9 Considering that experience with a software increases the likelihood that the individual will rate themselves as having a higher level of self-confidence and therefore think more positively of the technology and that individuals who think more highly of a technology are more likely to adopt it, it would make sense for training to provide the experience that would create the buy-in required for wide-spread adoption of the technology. 1 Best Practices, 2008 2 Nowak and Krcmar, 2003 3 Nowak and Krcmar, 2003
5 4 Hartzel, 2003 5 Kay and Loverock, 2007 6 Kay and Loverock, 2007 7 Co, Patuwo, and Hu, 2008 8 Curran and Mueter, 2005 9 Curran and Mueter, 2005 Methodology It is expected that this study will prove that training increases the comfort level, confidence level, and happiness level with the technology in question and therefore support the best practice rule of providing training to increase employee buy in and usage of required technology. In order to better examine the relationship between training, user self-confidence in his/her ability to use a technology, the emotional response to using that technology, and the actual usage of that technology, a study was set up to test the effect of training on user buy-in. In this study, user buy-in was measured by the user s selfrating of comfort level, confidence level, happiness level, and actual usage of the software. Participants would take a survey rating their confidence level in their ability to use the software and their emotional response to using the software. They would then take part in a training session on a particular piece of software followed by taking the second part of the survey asking similar questions on their confidence and emotional response. The actual usage of a piece of software was monitored in the week preceding and following the training. For the purpose of this study, an employee was an individual employed by IntraHealth International and working in the Knowledge Services or Communications Departments in February 2009. The software upon which this study was focused is IntraHealth International s internal SharePoint Staff News blog (a blog that is accessible
6 and editable by IntraHealth International employees only). The approximately 10 minute training session dedicated to explaining how to use the internal blog took place in IntraHealth International s Chapel Hill Headquarters 9 February 2009. Buy-in was defined by a combination of happiness with the required use of IntraHealth International s Staff News blog, confidence level in own ability to use IntraHealth International s Staff News blog, and frequency of use of IntraHealth International s Staff News blog. The happiness level was a self rating of positive feeling towards the mandatory use of a blog as expressed by user recorded in a written survey given immediately prior to and immediately following training session given at IntraHealth International s Chapel Hill Headquarters 9 February 2009. Confidence level was recorded as self rating of belief in self competency as expressed by user recorded in a written survey given immediately prior to and immediately following training session given at IntraHealth International s Chapel Hill Headquarters 9 February 2009. Usage was defined as any time a user adds a post to the Staff News blog within the two week period surrounding the training (2 February 2009-13 February 2009). The researcher modeled the training session on an existing IntraHealth International internal training on using SharePoint blogs. The two (2) page two (2) part survey asked questions regarding the participant s past experience and current level of comfort, knowledge, and satisfaction. Part A, which was taken prior to the training session, surveyed the participant s past experience with blogs, any previous training on blogs, their current level of comfort, knowledge, satisfaction, and frequency of use of the Staff News blog. Part B, which was taken immediately following the training session, included many of the same questions that were on part A including their current level of
7 comfort, knowledge, satisfaction, and frequency of use of the Staff News blog. Part B questions also covered what the participants thought of the training (if they think they would prefer a different type of training, what sort of follow up they would appreciate). On 2 February 2009, the researcher left a cover letter on each employee s desk (please see the cover letter attachment) introducing the study to the potential participant and asking them to come to the training session scheduled for the morning of 9 February 2009 if they are willing to participate. A consent form was included with the cover letter. An Outlook Meeting invitation with the same content of the cover letter was emailed to all members of the Communications and Knowledge Services Teams on 2 February 2009 in order to reserve the time and location on their calendar should they choose to accept it. On Monday 9 February 2009, an approximately thirty (30) minute long training session took place. The training session began with a brief introduction to the study and each participant received the two page two part questionnaire as well as the consent form. The consent form was collected before the first part of the survey was filled out. Each participant wrote their name on a sticky note and place it on the questionnaire to loosely link the survey number with the participant s name. Each participant then take part A of the survey. Following the survey, the participants then took part in the ten (10) minute training session while they follow along with their computers. They then took part B of the survey which indicated their new feelings of comfort, knowledge, satisfaction and expected frequency of use. After being thanked for their time, the training session was concluded. On 14 February 2009, the researcher recorded the number of changes made to the blog by study participants between 2-6 February 2009 and 9-13 February 2009.
8 Analysis Though everyone who attended the training session expressed increased confidence in their ability to use the SharePoint blog, there was only one individual who had no previous experience with a SharePoint blog. That same individual was the only participant who also answered the question does the thought of adding a post to a SharePoint blog worry you? positively in the questionnaire given before the training session. However, after the training, the participant s belief that the SharePoint blog was easy to use had switched from no (before the training) to yes (after the training) and the thought of adding a post to the blog no longer worried her. Additionally, the participant s self-rating of her confidence in her ability to use a SharePoint blog had increased and her emotional response has increased positively. Four people s self-ratings of their confidence in their ability to use a SharePoint blog had increased following the training. These increases were to be expected as foreshadowed by Nowak and Krcmar s previous work. Additionally, three people s emotional responses increased positively following the training; two of the participants emotional reactions increased by one and one of the participants emotional reaction increased by two from feeling neutral towards the software to loving it. While at first glance this appears to contradict research done by Kay and Loverock showing that long term training programs decreased negative emotions such as anxiety and fear but had no effect on happiness or sadness, it may be that this increase in positive feelings towards the software is a temporary increase which will subside in time. Additionally, it may merely be that a decrease in negative emotions may have shown up as an increase in positive emotions.
9 Interestingly, there were two cases of un-corresponding increases in confidence and positive emotional responses. That is, one individual whose confidence increased was not followed by a corresponding positive emotional increase. Another individual s increase in positive emotional response was not followed by a corresponding increase in confidence. This was not expected as it had been hypothesized that an increase in one would correspond to an increase in the other. Oddly, although two individuals believed they would post more often after the training, two individuals believed they would post less frequently following the training. However, there were no changes in number of actual postings by each individual before and after the training. Although no new postings occurred, the researcher received an email from one of the other participants mentioning that the training had inspired her to think about posting to the blog although an actual post had not occurred (personal communication). Another employee who participated in the study pointed out that even though the training had improved her outlook towards the blog and her belief in her ability to use it, she didn t have the time to come up with things to post (personal communication). Conclusions The training did improve the emotional response of some employees to the use of the blog as well as the confidence in the employee in the employee s ability to use the blog. There was no change in the usage of the blog: the only employee who posted after the training had also posted once before the training.
10 While the results of this study were not as astounding as the research could have hoped, the researcher believes that if this study were done in the future, several changes would result in improved results. Allowing for more time, such as a month, both before the training session and after the training session would provide the researcher with a more accurate image of usage of the blog. Additionally, although the research is incredibly grateful to IntraHealth International s management for its support and the Communications and Knowledge Services Teams for their participation, a larger selection of employees would encompass a more accurate selection of average SharePoint users. Although further testing may reveal more complete results, it appears that these results uphold the general belief that training increases at least part of user buy-in. As a result, it appears that requiring training is a good thing for businesses to incorporate in their plan of rolling out new technology if they wish their employees to feel more positive towards the new technology. More study needs to be done to determine if training increases the actual usage of the technology and to explore the rise and fall of positive and negative emotional reactions to the software over time. However, based upon research previously done as mentioned above, it seems that training does increase the usage of a technology and therefore supports the belief that training increases user buy-in.
11 Works Cited Best Practices, LLC. Deploy customized SFA training. Chapel Hill, NC: Best Practices, LLC, 2008. Best Practices, LLC. Employee Training Tactics for Successful Order Management System Implementation. Chapel Hill, NC: Best Practices, LLC, 2008. Beukelman DR, Hanson E, Hiatt E, Fager S, Bilyeu D. AAC Technology Learning Part 3: Regular AAC Team Members. Augmentative and Alternative Communication. 2005;21:3, Pages 187-194. Burke R, Beukelman DR, Ball L, Horn CA. Augmentative and Alternative Communication Technology Learning part 1: Augmentative and alternative communication intervention specialist. Augmentative and Alternative Communication. 2002;18:1, Pages 242-230. Co HC, Patuwo BE, Hu MY. The human factor in advanced manufacturing technology adoption: An empirical analysis. International Journal of Operations and Production Management. 2008;18:1, Pages 87-106. Curran JM, Meuter ML. Self-service technology adoption: comparing three technologies. Journal of Services Marketing. 2005;19:2, Pages 103-113. Hartzel K. How Self-Efficacy and Gender Issues Affect Software Adoption and Use. Communications of the ACM. 2003;46:9, pages 167-171. Kay RH. Exploring the relationship between emotions and the acquisition of computer knowledge. Computers & Education. 2008;50:4, Pages 1269-1283. Kay RH. Learning performance and computer software: an exploration of knowledge transfer. Computers in Human Behavior. 2007;23, pages 333-352. Kay RH, Loverock S. Assessing emotions related to learning new software: The computer emotion scale. Computers in Human Behavior. 2008;24:4, Pages 1605-1623. Nowak K, Krcmar M. Improving computer efficacy and perceptions of technology related careers in the classroom: A case study. Conference Papers -- International Communication Association; 2003 Annual Meeting, San Diego, CA, p1-25.