Aaron Kim and Jean-Marc Wise Florida State University

Transcription

1 Measuring College Students Perceptions and Attitudes toward Anti- Detection Tools and Their Behaviors, Beliefs, and Moral Judgment regarding Aaron Kim and Jean-Marc Wise Florida State University Abstract The focus of this study is on measuring the impact of using anti-plagiarism tools (APT s) on students behavior and attitudes toward plagiarism. A survey instrument was developed and validated in order to collect self-reported data on perceptions/attitudes, behavior, beliefs, and moral judgment with respect to plagiarism and the use of APT s. The survey was deployed with an undergraduate section at a southeastern research I university, yielding a response rate of 72.3% (n=133). Results found significant correlations between APT perception and plagiarism behavior, moral judgment, and neutralization (i.e. justification of unethical behavior). Findings also confirmed previous research with respect to correlations between attitudes, moral judgment, and plagiarism behavior. Details on survey development and validity, as well as suggestions for future research are provided. Literature Review It is fair to say that plagiarism may threaten the integrity of higher education in colleges and universities throughout the U.S. About one in two students have cheated on tests and 71% and 85% of college students who participated in a study reported plagiarism and cheating on assignments, respectively (Stephens et al, 2007). Although Stephens defines cheating and plagiarism rather widely, figures from other studies are also staggering. For example, 70% of 18,000 high school students admitted they had cheated; and 60% plagiarized (Villano, 2006). Even more alarmingly, most of the students reported in Villano s article do not consider cheating as wrong and they feel justified in their behavior. They don t understand why they should be held to a higher standard said McCabe, who has studied the field of academic integrity for decades, in an interview with Slobogin from CNN (Slobogin 2002). The report is consistent with the findings from Stephens study in which the students who reported cheating and plagiarizing showed a lower sense of moral responsibility and a higher tendency of neutralization or justification of their behaviors than non-cheating students (Stephens 2001). Students behaviors and moral sense have been further investigated by Lovett-Hooper et al. They looked at the relationship between academic dishonesty and norm/rule-violating behaviors (Lovett-Hooper et al., 2007). The study found a positive relationship between all three Academic Dishonesty subscales (self-dishonest, social falsifying, and plagiarism) and an imagined norm/ruleviolating future. In short, the study suggests academic dishonesty may lead to rule-violating behavior in the future. In 2002, the Association of American Colleges and Universities (AAC&U) published a report on higher education of the 21st century to suggest comprehensive principles and models that improve learning for all undergraduate students entitled Greater Expectations: A New Vision for Learning as a Nation Goes to College (AAC&U, 2002). It envisions what society expects from college education and identifies three characteristics of learners as being empowered, informed, and responsible. Among dishonest behaviors, plagiarism has become a focal point recently as information technology advances and the use of the Internet is commonplace. In surveys run from 2002 through 2005, 38% of 18,000 college-aged students said they had cut-and-pasted from the Internet, 25% of graduate students also reported engaging in plagiarism, and 80% faculty observed plagiarism (McCabe, 2005). In an effort to fight plagiarism, many institutions put in place anti- plagiarism tools (APT) such as an online plagiarism detection service. There exist several commercial services including EVE2 (Easy Verification Engine: canexus.com/eve/), Glatt Services ( plagiarism.com/), SafeAssignment ( ), and TurnItIn (htpp:// These tools are generally well received by students because of the convenience of online submission, and by instructors as an additional tool to address the plagiarism issue. But only few studies have evaluated such tools systematically. In a 2006 study, TurnItIn was evaluated in terms of how the system works, user experiences, and data generated by the system (Evans, 2006). The author suggestes that the service detects problematic practices but it may not be practical to check every script thoroughly. The verdict was working but impractical. The growing popularity of these tools is likely due to a variety of reasons. For one, there seems to be a general agreement among instructors that awareness of such a tool can be a deterring factor; students do not risk getting caught if such a tool is in place. As shown in a recent study by Dahl (Dahl, 2007) one of the main advantages of 256

2 APT s may be their psychological effect. Awareness of TurnItIn among the participants in the study increased awareness of academic integrity and plagiarism (e.g., what is considered plagiarism, honor code, etc). The awareness also clearly brought about some behavioral changes: More than half of the participants learned how to reference correctly after using TunrItIn, which tells us that such a tool can also have educational benefits for higher academic standard (Dahl, 2007). Therefore, it is important to examine how using APTs can affect students behaviors, attitudes, and perceptions with respect to ethical conduct. What psychological effects and educational benefits of using APT s can bring is an important question. Thus, in this current study we investigate the following questions: 1. What are college students awareness, perceptions and attitudes, and beliefs with respect to anti-plagiarism tools? 2. What is the relationship between perceptions/attitudes toward plagiarism, plagiarism behavior, moral judgment and beliefs, and perceptions/attitudes toward anti-plagiarism tools? 3. What are the effects of using anti-plagiarism tools? 4. What is the relationship between effects of using APT s and other variables of interest? To answer the research questions, a self-report questionnaire was developed and used to collect data. Correlational analysis was the main method used to examine the relationships among variables based on the students ratings on the corresponding statements. The questionnaire included statements related to specific effects and benefits of APT s suggested in the literature as an attempt to address the questions. The statements include After I learned my instructor was using an APT, I still plagiarized and I decided not to plagiarize (deterrence effect); I became more aware of plagiarism (awareness effect); I tried to find out how to avoid committing plagiarism, I tried to learn how I should reference something correctly because I experienced an APT and I feel confident about quoting other people s work correctly (educational benefit). Other variables operational definitions are as follows. Variables of interest awareness is the level of students awareness or knowledge about what constitutes plagiarism. behavior represents how many times a student engaged in plagiarism. Perception/attitude on plagiarism includes three subsets: moral judgment, belief (neutralization), and plagiarism perception. In a 1991 study, moral judgment was shown to have a strong relationship with college students dishonest behavior (i.e., cheating on exam) and suggested a strong predictor of such behaviors (Beck & Ajzen, 1991). The study suggested that students with high moral judgment level are less likely to engage in dishonest acts. In another study, neutralization was also shown as a strong predictor of academic dishonesty (Diekoff et al., 1996). Neutralization means students tendency to justify dishonest behaviors attributing external and internal factors. Thus, the study suggested that students with high neutralization level are more likely to engage in dishonest acts. Perception/attitude of APT s is about how students feel about APT s, whether they support the use of APT s, feel intimidated, see APT s as a reliable tool, and so forth. Method Participants Participants were 184 students enrolled in a non-major course in southeastern public university. The researchers decided to use one course that is representative of the university population and work with the instructor of the course to ensure reasonable response rate. The instructor of the course gave extra credit to the students who completed the survey. The response rate was 72.3% (133 students participated). Since the survey should be anonymous, the researcher assured the students that the fact of their participation would be stored separately from the survey data and that their responses would not be associated with their identity. The respondents consisted of 6 freshmen (4.5%), 55 sophomores (41.4%), 56 juniors (42.1%), and 15 seniors (11.3%). 64% of the respondents were female. The majority of the students major in Communications Studies (42%) and other majors included English, Marketing, Media Production, Business, Public Relations, Political Science, and International Affairs. Procedures The first announcement regarding the study and the procedure was made during the class in September 2009 and was followed by an advertisement and announcement on the course website. All students enrolled in the course were ed with a link to the online survey along with a brief summary about the study. Anonymity of 257

3 participation was emphasized in all announcements and individual inquiries regarding anonymity and procedures were answered during the class visit and via s by the researchers and the instructor. Once the students opened the questionnaire, they needed to provide their informed consent in order to proceed. The online questionnaire was available for one week. In addition to the announcements in class and on the course site, and the initial invitation, a reminder was sent out on the third day and another one on fifth day of the data collection. Measures A survey instrument was developed for this study, which included newly developed items and other items that were adapted from the existing instruments. It is a self-report questionnaire that consists of forty five statements, most of which are rated using 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), and eleven statements with slightly different scales. Please see Appendix A for a copy of the instrument. New statements were written in order to measure the variables of interest: the students awareness, perceptions and attitudes, beliefs, and morality with respect to plagiarism and APT s. Subscales of moral judgment and beliefs were adapted from a moral responsibility scale (Beck & Ajzen, 1991) and a neutralization scale (Diekoff et al., 1996). Students academic dishonesty was measured by items adapted from the Academic Dishonesty Student Survey (McCabe, 1992). Structure of the measurement instrument The measurement instrument consists of 5 sections measuring the variables of interest in addition to APT experience (section 4) and the demographic information (section 7). The following Table 1 summarizes the structure of the instrument with the different sections and associated variables. Table 1. Structure of the Measurement Instrument Section Variables Subscale (if any) Scale Variable type 1. Awareness Disagree, Not sure, and Agree. Interval (Pretest total score) 2. Behavior Never, 0, to More than three Interval times, 5 3. Perception/Attitude toward - Moral judgment - Neutralization 5-point Likert scale ranging from 1 (strongly disagree) to Interval 4. Anti- Tool (APT) Experience - perception - APT Awareness - Experience 5 (strongly agree). - Yes, No - Number of courses and instructors - Categorical - Interval 5. Perception/attitude of Anti- 5-point Likert scale ranging Interval Tool (APT) 6. Effects of APT - Educational Benefits from 1 (strongly disagree) to 5 (strongly agree) with N/A Interval option 7. Demographic Information Categorical Section 2 uses a modified version of McCabe's Academic Dishonesty Student Survey (1992). Most statements in the section 3 were developed in accordance with items in sections 5 and 6. Section 3, perception/attitude on plagiarism, contains subscales of moral judgment and beliefs that were adapted from a moral responsibility scale (Beck and Ajzen, 1991) and neutralization scale (Diekoff et al. 1996). Development of the measurement instrument Newly developed items were drawn from the literature and tested and revised through a panel review, an expert review, and formative evaluation. The instrument with the new items was reviewed by twelve doctoral students who are experienced in survey research and instrument development. The revised instrument then was reviewed by an expert who has been conducting and teaching survey research methods at a Research I University for more than 20 years. Formative evaluation involved four representative students with different majors and years in school. During the formative evaluation session students engaged in open discussion and critique of the survey instrument (e.g., clarity of statements, overall structure, effective communication, etc). The instrument was once again revised based on the result of the formative evaluation. 258

4 Validity and reliability test Formative evaluation provided opportunities for assessing not only the quality of items but also content validity. Since there is no external criterion that can be compared to in terms of plagiarism behavior, perception, and morality, criterion validation is beyond the scope of this study. Establishing construct validity, however, may be possible for some items that were modified from the existing items. The authors made a judgment based on how the results of this study relate to the previous studies that used the same instrument. For example, the instrument measuring moral judgment and beliefs revealed that plagiarism is highly correlated with moral judgment and beliefs of students (Beck & Ajzen, 1991). Thus, construct validity of the modified items of moral judgment and belief subscales can be assessed in addition to the content validity. The reliability estimates for each section and subscales are summarized in Table 2. Table 2. Reliability Estimates Section Variables N of Items Cronbach's Alpha 1. Awareness 6 N/A a test score 2. Behavior Perception/attitude on a. - Moral judgment b. - Neutralization c. - perception 11 (4)* (2) (5).693 (.660) (.440) (.737) 4. Anti- Tool (APT) Experiences 5 N/A** 5. Perception/attitude of Anti Tool (APT) 6. Effects of APT Demographic Information 6 N/A *subscales in parenthesis **2 questions pertains to the number of courses and instructors the students counted as experiences with APT and plagiarism discussion. Results College students awareness, perceptions and attitudes, and beliefs with respect to anti-plagiarism tools The awareness level was measured by six questions each asking whether a specific act was a type of plagiarism. The majority of the students scored 3 or less, which may indicate the lack of knowledge about plagiarism among the students. Moral judgment, plagiarism perception, neutralization, and APT perception are all quite high. APT effects showed high agreement that their experiences with APT affected their behaviors and perception. The following descriptive statistics table summarizes the results. Table 3. Descriptive Statistics Range Std. Skewness N Min Max Mean Deviation Statistic Std. Error Awareness Moral Judgment Perception Neutralization APT Perception Data Preparation for statistical analysis Preliminary data examination showed that there were some missing cases and undifferentiated responses. Further examination of such cases convinced the authors to drop the undifferentiated cases from the analysis. A total of 10 cases was removed from the data set because they either showed no variance in their answers to one or more sections of the questionnaire or provided answers that did not match when they were supposed to match such as reverse coded items. In order to address the issue of missing data, we calculated the percent score for some variables instead of raw score using only answered items. Since there were only a few missing values and the missing values should not affect the overall score, a raw score divided by the maximum possible score would best represent the measures. 259

5 Perception/attitude on plagiarism (moral judgment, belief (neutralization), APT effects, and plagiarism perception used percentage scores while plagiarism awareness and behavior used raw scores. Correlation Analysis This study examined the correlations among moral judgment, beliefs (in the neutralization subscale), plagiarism behaviors, and attitudes/perceptions toward anti-plagiarism tools (APT s). The following correlational analysis table summarizes the results of this study. It shows relationships among variables and the effects of APT use. Educational benefit is a subscale under APT effects that included items such as I became more aware of plagiarism ; I tried to find out how to avoid committing plagiarism ; I would prefer to get more guidance about what plagiarism is ; I tried to learn how I should reference something correctly because I experienced an APT. Table 3. Correlations Awrns Plagiaris Moral Neutralizatio APT m Judgment Perception n Perception Awareness Behavior (**) (**) -.310(**) Moral Judgment 1.209(*) -.429(**).307(**) Perception (*).158 Neutralization (**) APT Perception 1 APT Effects Ed. benefit Experience ** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). Moral judgment and perception of APT are negatively correlated with plagiarism behavior while neutralization is positively correlated. behaviors represent the frequency of dishonest behaviors that the students reported. Thus, the study confirmed that the higher their moral judgment, the less likely it is for the students to involve in plagiarism activities. Likewise, the more the students have a positive perception of APT s, the less likely to plagiarize. On the other hand, the more students tend to justify their dishonest behavior, the more likely they are to plagiarize. Perception of APT correlates positively with moral judgment and correlates negatively with neutralization, which could mean that those students who perceive plagiarism negatively tend to avoid plagiarism and perceive APT s positively. APT perception is also positively correlated with APT effects. In other words, those who see APT s positively tend to attribute their specific perception and behavioral changes to their experience with APT s, leading to a higher plagiarism awareness level or a decision not to plagiarize. APT effects Based on the students responses, the majority of the students agreed that APT use affected their behavior and awareness with respect to plagiarism. Table 4 summarizes the students responses to the APT effects statements and Table 5 summarizes the correlation between APT effects and other variables. Table 4. APT Effects N/A Strongly disagree Disagree Neutral Agree Strongly agree Total % 52.0% 22.8% 4.9% % 4.1% 8.9% 15.4% 55.3% 9.8% % 4.1% 4.1% 18.7% 35.0% 13.0%

6 12.2% 8.9% 8.1% 23.6% 40.7% 6.5% % 13.0% 33.3% 35.0% 16.3% 0.8% % 1.6% 5.7% 14.6% 65.0% 12.2% % 6.5% 17.9% 22.0% 34.1% 4.9% *1. I still plagiarized; 2. I became more aware of plagiarism; 3. I decided not to plagiarize; 4. I tried to find out how to avoid committing plagiarism; 5. I would prefer to get more guidance about what plagiarism is; 6. I feel confident about quoting other people s work correctly; 7.I tried to learn how I should reference something correctly because I experienced an APT. APT effects are deterrence effect (statement 1 and 3), awareness effect (statement 2), and educational benefit (statement 4, 5, 6, and 7) Table 5. Correlation Analysis with APT Effects Awareness Plagiaris Moral m Neutralization APT Judgment Perception Behavior Perception Awareness (*) Behavior (**) (**) -.293(**) Moral Judgment.209(*) -.278(**) (**).313(**) Perception (*).128 Neutralization (**) -.372(**) -.206(*) (**) APT Perception (**).313(**) (**) 1 APT Effect 1* (**) (*) (*) (**).212(*) (**).377(**) (*) *Analyzed APT effects that did not include n/a responses. Students plagiarism behavior was directly reflected on the first statement of APT effects and the correlation between plagiarism behavior and APT s deterrence effect was positive and significant. Likewise, students awareness level was positively correlated with APT s awareness effect item. Other significant correlations were found between one of the APT s educational benefit item ( I feel confident about quoting other people s work correctly ) and plagiarism behavior, moral judgment, neutralization, and APT perception/attitude, respectively. 261

7 Discussion The research questions of this study were: 1. What are college students awareness, perceptions and attitudes, and beliefs with respect to anti-plagiarism tools? 2. What is the relationship between perceptions/attitudes toward plagiarism, plagiarism behavior, moral judgment and beliefs, and perceptions/attitudes toward anti-plagiarism tools? 3. What are the effects of using anti-plagiarism tools? 4. What is the relationship between effects of using APT s and other variables of interest? With the first two questions we aimed at gaining a better understanding of the factors that contribute to plagiarism behavior in students. As presented in the results section, there is positive relationship between moral judgment, and APT perception/attitude. On the other hand, neutralization and plagiarism behaviors have a negative relationship. The third and forth questions, however, could not be answered as expected since there is not enough information available, even though some correlations turned out significant (i.e., moral judgment and overall APT effect score, r =.184, p <.05; APT perception and APT effect score, r =.253, p <.01). For example, educational and other benefits of using APT s was a main variable of interest in this study but could not be tested appropriately given the data since there were only 8 students who have never experienced an APT before whereas the rest of the students have experienced it. Although correlations are statistically significant, this may not be a meaningful result because the data set violates the normality and equal variance assumptions that need to be met to run a statistical test. Since most students already knew about APT s and have used at least one before, there must be other ways to differentiate their experience with APT s or a large enough number of non-experienced users should be surveyed. College students awareness, perceptions and attitudes, and APT perception/attitude In discussing plagiarism awareness, perceptions and attitudes, and APT perception and the corresponding analysis, it is important to note that most of the students who participated in the study knew about APT s and had experienced APT s before. Only eight students indicated that they didn t know about APT s and had never taken a course that used APT s. Interestingly, the students awareness level turned out low although their self-reported perception of plagiarism and moral judgment level were high. This may suggest that they don t know what constitutes plagiarism but they are aware that plagiarism is an issue and can have serious consequences. The absence of an association between behavior and awareness was confirmed by analysis of correlation. Also plagiarism awareness is not associated with moral judgment, plagiarism perception, neutralization, or APT perception, which makes sense because knowing about plagiarism does not necessarily mean high level of moral judgment and other variables. APT perception/attitude was measured in this study along with other variables that were found in other academic dishonesty studies in order to examine the relationship with those variables (e.g., moral judgment and dishonest behaviors). APT perception/attitude was found to have a positive relationship with moral judgment while having a negative relationship with plagiarism behavior and neutralization. It seems valid and reasonable to include APT perception/attitude as a main variable in plagiarism research. This study shows results that are consistent with previous studies in addition to presenting new findings from including additional variables such as APT perception/attitude. For example, the measure of moral judgment and beliefs confirmed that plagiarism behavior is highly correlated with moral judgment and beliefs of students (Beck & Ajzen, 1991). In terms of validity of the instrument developed for this study, consistent results and correlations among major variables strongly support the construct s validity with respect to the modified items as well as the newly written items. In addition to confirming findings of prior research, the present study shows that the use of APT s is related to how students behave and how they perceive plagiarism. The fact that this study found significant relationships between the use of APT s and other major variables, such as moral judgment, suggests that APT s should be included as a major variable in future research on plagiarism. APT effects First, as to the deterrence effect (statement 1 and 3), none of the respondents stated that they plagiarized and about 50% of them indicated they decided not to plagiarize after they learned that APT s were used. It is particularly important to note that about 20% said that the deterrence effect was not applicable to them. During the formative evaluation, all of the students pointed out that the not applicable option was needed because they did not plagiarize and decided not to do so regardless of APT use. It may not be reasonable to make a conclusive remark 262

8 about not applicable but helpful to understand and the difference between agree and disagree responses. In other words, by excluding not applicable responses, agreement with the deterrence effects of APT can become clearer. The awareness effect (statement 2, I became more aware of plagiarism ) was also clearly shown to be one of the main APT effects. Using an APT raised the level of plagiarism awareness, which was also consistent with students high plagiarism perception score (mean = 20.67, sd = 2.75, max = 25). The APT awareness effect is also supported by the correlations with plagiarism awareness and moral judgment although the size of the correlations is quite small. On the other hand, plagiarism perception and APT awareness effect are not associated based on the correlation analysis. behavior and APT perception are not associated either with APT awareness effect as it is not associated with plagiarism awareness. Most educational benefit items also did not produce significant correlations with the other variables. Only the responses to the statement 6 ( I feel confident about quoting other people s work correctly ) turned out to have significant correlations with plagiarism behavior, moral judgment, neutralization, and APT perception. About 77% of the respondents agreed with the statement, which may indicate benefit of using APT s. In fact, many of the respondents agreed with other educational benefit statements except the statement 5, which was about whether they wanted more guidance or not. If they were already confident as indicated in their response to the statement 6, they would not seek more help. In their responses to statement 4 and 7, they indicated they tried to learn more about avoiding plagiarism (about 50% of the respondents agreed excluding n/a responses). Despite the absence of significant correlation, educational benefits seemed to be clearly shown. Limitation of the Study This study involved the collection of sensitive information from students by means of an anonymous webbased survey. In order to encourage participation, the survey was deployed in the context of an undergraduate course and students were able to earn participation points toward their grade as a reward for participating in the study. Although the students anonymity was guaranteed, it is possible that some students were reluctant to admit to any unethical behavior they may have engaged in, which might explain the comparatively low scores on that variable in contrast to other studies of undergraduate students. One might argue that self-reports should be considered with caution in general, because participants might have a distorted perspective of their own behavior; they might report how they think they should behave or how they would like to behave, rather than how they actually behave. Additionally, the vast majority of the participants had had some experience with ATP s in the past; this prevented us from effectively comparing the responses between experienced and inexperienced students and as a consequence it was not possible to establish any differences between the groups. A larger sample size would likely yield a better distribution and thus offer additional opportunities for analysis. Recommendations for Future Research Current study only surveyed 133 students, which is an intact group to ensure higher response rate and achieve good representativeness. Despite the high response rate, the intact group participation unexpectedly resulted in too small number of students who had not experienced APT prior to the data collection. It is crucial to have enough number of non-experienced participants to compare variables based on the student s experiences with APT. Thus, more participation would be desirable in a future research. Also it would be ideal to have any way to complement and support the findings from self-report questionnaire method. Observation (e.g., examining actual writing assignments) would provide invaluable insights and potential verification of students responses. Since it can be hypothesized that there are differences in the variables of interest depending on levels of awareness of and perception/attitude toward APT s, it may prove beneficial to employ MANOVA in order to investigate the effects of awareness and interactions among key variables (e.g., morality, awareness, and attitude). 263

9 References Beck, L.,&Ajzen, I. (1991). Predicting dishonest actions using the theory of planned behavior. Journal of Research in Personality, 25, Dahl, S. (2007). Turnitin[R]: The student perspective on using plagiarism detection software. Active Learning in Higher Education, 8(2), Diekoff, G. M., LaBeff, E. E., Clark, R. E.,Williams, L. E., Francis, B., & Haines, V. J. (1996). College cheating: Ten years later. Research in Higher Education, 37, Evans, R. (2006). Evaluating an electronic plagiarism detection service: The importance of trust and the difficulty of proving students don't cheat. Active Learning in Higher Education: The Journal of the Institute for Learning and Teaching, 7(1), Lovett-Hooper, G., Komarraju, M., Weston, R., & Dollinger, S. J. (2007). Is plagiarism a forerunner of other deviance? Imagined futures of academically dishonest students. Ethics & Behavior, 17(3), McCabe, D. L. (1992). The influence of situational ethics on cheating among college students. SociologicalInquiry, 62, McCabe, D. L. (2005). Cheating among college and university students: A North American perspective. International Journal for Educational Integrity, 1(1). McKeever, L. (2006). Online plagiarism detection services saviour or scourge? Assessment & Evaluation in Higher Education, 31(2), Slobogin, K. (2002). Survey: Many students say cheating's OK. Retrieved on 2/14/09 from Stephens, A. M., Young, M. F., & Calabrese, T. (2007). Does moral judgment go offline when students are online? A comparative analysis of undergraduates beliefs and behaviors related to conventional and digital cheating. Ethics & Behavior, 17(3), Villano, M. (2006). Taking the work out of homework: with the rise of the internet, schools are seeing an epidemic of cut-and-paste plagiarism. But the same technology that's making plagiarism easy is being used by teachers to catch copycats in the act (fighting plagiarism). T H E Journal (Technological Horizons In Education) 33(15),