Distribution Trend of the Information Skills for Students Based on Visual Analog Scale and Likert Scale

Transcription

1 IJBSCHS [Original Article] Biomedical Soft Computing and Human Sciences, Vol.20, No.2, pp.1-6 CopyrightC1995 Biomedical Fuzzy Systems Association (Accepted on ) Distribution Trend of the Information Skills for Students Based on Visual Analog Scale and Likert Scale Satoshi WATANABE*, Naruki SHIRAHAMA**, Yuji MATSUMOTO*, Hiroyuki TSUKAMOTO*, Masafumi NAKAGAWA***, Kazunori MIYAMOTO****, Naofumi NAKAYA*****, Masashi TOMITA****** and Yukio MORI****** *Department of Information Design, School of Information Studies, Shizuoka Sangyo University **Department of Electronics and Control, National Institute of Technology, Kitakyushu College ***International University of Health and Welfare Hospital ****Department of Human Development, Faculty of Humanities, Kyushu Women s University *****Department of Electronic Technology, Tokyo Electronics College ******Department of Mechanical and Electronic Engineering, Salesian Polytechnic Abstract: This paper relates the distribution trend of the information skills for students based on Visual Analog Scale (VAS) and Likert scale (LS). The data are collected from 40 subjects. They were invited to answer LS questionnaires first, and then to answer VAS questionnaires a week later. To apply the same evaluation criteria, VAS data and LS data are transformed to ratio scales value (i.e to 1.00) and the subjective evaluation distribution based on LS and VAS are calculated. The experiment result shows that there exists unintended bias between LS data and VAS data. This fact is supported by the published articles. Therefore, it can be considered again that the presence of bias is affected by the difference of questionnaire. To obtain reliable conclusions, it is necessary to try this method on the results of feature study and do more investigation. Keywords Subjective evaluation, Distribution, Visual analog scale, Likert scale. 1. Introduction Studies on the subjective evaluation based on Likert Scale (LS) [1] have been conducted broadly. LS is a simple discrete scale method, but it has some weak points as follows [1-5]: 1) LS s evaluation often becomes complicated; 2) Sometimes several bias are introduced by the following factors, such as halo effect, leniency effect, and central tendency; 3) LS s results are discrete data. From these points, it is suggested that there exists unintended bias on LS s results. In order to solve these weak points, Visual Analog Scale (VAS) is investigated on the subjective evaluation [6-7]. VAS is a ratio scale method, it is expected that it can solve the weak points of LS as mentioned above. And it is considered that VAS is more flexible than LS. This paper relates the distribution trend of the information skills for students based on VAS and LS. As Suruga-dai, Fujieda-shi, Shizuoka, Japan Phone: watanabe@ssu.ac.jp 1 for LS, it employs Likert s simplified method [1] [8], which has an equal interval of the scale. To apply the same evaluation criteria, VAS data and LS data are transformed to ratio scales value (i.e to 1.00) and then subjective evaluation distribution is calculated. 2. Subjective Evaluation Based on VAS and LS Recently, VAS has been applied to study the subjective evaluation such as medical field [9-11], education [6-7], [12]. This paper introduces VAS in comparison with LS. Fig.1 shows typical types of LS s questionnaire. This is to ask the information skills. To answer the questionnaire in Fig.1, the response is marked as a discrete value from 1 to 5 on LS (only one place). The same questionnaire in VAS, is shown in Fig.2. To answer Fig. 2, the response is marked by a solid dot on the line (Example is shown in Fig.4).

2 Biomedical Soft Computing and Human Sciences, Vol.20, No.2, 2016 WP* Word Processor Spreadsheet Software Fig. 1. Typical Types of LS s Questionnaire Fig. 3. Transformation of LS Data: the location with the circle (only one place), convert to the following values (determined to only one value in 0.00~1.00). *WP: Word Processor Word Processor (a) 4.50cm SS** (b) 10.0cm Result = 4.50 / 10.0 = 0.45 (or 45%) Spreadsheet Software Fig. 2. Typical Types of VAS s Questionnaire Fig. 4. Example of Calculation of VAS Data: it is determined by the ratio of (a) to (b) (only one value in 0.00~1.00). **SS: Spreadsheet Software To apply the same evaluation criteria, VAS data and LS data [1] [8] are transformed to ratio scales value (i.e to 1.00). Fig. 3 shows Transformation of LS data. This paper will convert the numbers marked with circle (only one place in 1-5) to the values as shown in Fig.3. Fig.4 shows an example of calculation of VAS data. VAS s results are determined by the ratio of (a) to (b) (in other words, it is determined by the position of the mark on the line). 3. Materials and Methods 3.1 Subjects The data are collected from 40 subjects. They were invited to answer LS questionnaires first, and then to answer VAS questionnaires a week later. Ethical considerations are as follows: The explanation of this study was given by the oral and documents to the subjects, and it was also given that participation in this study was voluntary. Furthermore, the data have been collected anonymously, and have been only used carefully for the presentation of this study. 3.2 Investigation of Subjective Evaluation It was carried out the following investigation of subjective evaluation based on VAS or LS. Table 1 shows these questionnaires. Questionnaire Q1 and Q2 are shown in Fig.1 (LS) and Fig.2 (VAS). And Q3 to Q8 are as same as Fig.1 or Fig2. All subjects were invited to answer LS questionnaires first, and then to answer VAS questionnaires a week later. 4. Results and Discussion To apply the same evaluation criteria, VAS data and LS data are transformed to ratio scales value (i.e to 1.00). Table 2 shows the result of Questionnaire of Q1 to Q8 (mean ± standard deviation, F value and t-test). From F-test s results, they are shown to be equal variances (p>0.05) on obtained data of all questionnaires. 2

3 S. WATANABE, et al.: Distribution Trend of the Information Skills for Students Based on Visual Analog Scale and Likert Scale Table1. Questionnaires Based on LS and VAS Question No. Question: Are you good at...? Q1 Word processor operation? Q2 Spreadsheet software operation? Q3 Presentation software operation? Q4 Searching on the internet? Q5 Making of web pages? Q6 Programing operation? Q7 Image processing operation? Q8 Computing? Table 2. Comparison of Mean Score of LS and VAS Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 LS 0.66±0.27* 0.43± ± ± ± ± ± ±0.23 VAS 0.60± ± ± ± ± ± ± ±0.23 F value 1.41** t-test n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. *mean±standard deviation, **F(39,39), n.s.: not significant The results show that there is no significant difference in all questions, between LS and VAS, from t-test s result. From these results, the subjective evaluation distribution is calculated. And in order to visualize, it is supposed that subjective evaluation distribution on VAS follows the subjective evaluation distribution on LS. First, the box-and-whisker plot [13] of subjective evaluation distribution on VAS is shown. Next, they are plotted vs. subjective evaluation distribution on LS. These plots are summarized in Fig.5. In this paper, all questionnaires are to answer subject s information skills, such as Are you good at word processor operation?. And one fact can be observed from Fig.5. It shows that there exists unintended bias between LS data and VAS data for the questionnaire. This fact is supported from the results in Table 2 and published articles [6] [14]. Therefore, it can be considered again that the presence of bias is affected by the difference of questionnaire [6]. To obtain reliable conclusions, it is necessary to try this method on the results feature study and do more investigation [6]. 5. Conclusion This paper relates the distribution trend of the information skills for students based on Visual Analog Scale (VAS) and Likert scale (LS). The data are collected from 40 subjects. Subjects were invited to answer LS questionnaires first, and then to answer VAS questionnaires a week later. To apply the same evaluation criteria, VAS data and LS data are transformed to ratio scales value (i.e to 1.00) and the subjective evaluation distribution is calculated. The experiment result shows that there exists unintended bias between LS data and VAS data. These differences are supported from the results in Table 2 and in published articles [6] [14]. Therefore, the presence of bias, it is affected by the difference of questionnaire, can be considered again [6]. And in order to obtain reliable conclusions, to try this method on the results feature study and do more investigation are very important [6]. 6. Acknowledgment This work is partially supported by JSPS Grants-in-Aid for Scientific Research, No The authors are heartily grateful to Professor Fenghui Yao of Tennessee State University for his kindness and encouragement. 3

4 Biomedical Soft Computing and Human Sciences, Vol.20, No.2, Q1 Q2 Q Q4 Q5 Q Q7 Q8 Fig.5. Subjective Evaluation Distribution on LS vs. Box-and-Whisker Plots [13] of Subjective Evaluation Distribution on VAS. Function y=x is expressed by the solid line in each of these plots. : outlier 4

5 S. WATANABE, et al.: Distribution Trend of the Information Skills for Students Based on Visual Analog Scale and Likert Scale References [1] Likert, R. A.: Technique for the Measurement of Attitudes, Archives of Psychology (New York), [2] Peabody, D.: Two components in bipolar scales: Direction and extremeness, Psychological Review, vol.69, No.2, pp.65-73, [3] Upshaw, H. S.: Own attitude as an anchor in equal-appearing intervals, The Journal of Abnormal and Social Psychology, Vol.64, No.2, pp.85-96, [4] Cummins, R. A.: The second approximation to an international standard for life satisfaction. Social Indicators Research, Vol.43 No.3, pp , [5] Heine, S. J., Lehman, D. R, Peng, K. and Greenholtz, J.: What's wrong with cross-cultural comparisons of subjective Likert scales?: The reference-group effect, Journal of personality and social psychology, Vol.82, No.6, pp , [6] Watanabe, S., Shirahama, N., Tsukamoto, H., Matsumoto, Y., Nakagawa, M., Miyamoto, K., Nakaya, N., Tomita, M. and Mori Y.:Trend of the Subjective Evaluation based on Visual Analog Scale and Likert Scale, Proceedings of 10th International Conference on Innovative Computing, Information and Control (ICICIC 2015), pp , August, 2015, Dalian, China. [7] Watanabe, S. and Matsumoto Y.: An Analysis of Information Skills for Students Based on a Visual Analog Scale, Journal of Biomedical Fuzzy Systems Association, Vol.13, No.1, pp.57-62, 2011 (in Japanese). [8] Murao, H.: Using Likert-type Item Data as Interval Data, Aomori Public College Journal of Management & Economics, Vol.10, No.2, pp.3-20, 2005 (in Japanese). [9] Inaoka, M., Yonenobu, S., Yamamoto, T. and Tada, K.: Risk factors associated with occupational low back pain. A prospective study: industrial vs. educational settings, The Journal of Japanese Society of Lumbar Spine Disorders, Vol.7, No.1, pp.79-88, 2001 (in Japanese). [10] Hanari, T. and Takahashi, S.: Effect of cognitive task on color preference measured by visual analog scale: Using like/dislike-activation task, Journal of the Color Science Association of Japan, Vol.28, pp.48-49, 2004 (in Japanese). [11] Naruhashi, K., Matsushita, R., Shimizu, S., Yamada, K., Miyamoto, K. and Kimura, K.: Evaluation by Students and Preceptors of Pharmacy Experience and Lectures in a Master Program Using Visual Analog Scale Method, Iryo Yakugaku (Japanese Journal of Pharmaceutical Health Care and Sciences), Vol.32, No.9, pp , 2006 (in Japanese). [12] Sagara, H., Nawa, H., Sendo, T. and Gomita, Y.: Comparison of the Visual Analog Scale Method and 5-Point Evaluation in Student Self-assessment of Comprehension and Acquisition in a Model Core Curriculum for Practical Training, Yakugaku Zasshi (Journal of the Pharmaceutical Society of Japan), Vol.127, No.4, pp , 2007 (in Japanese). [13] Online Statistics Education: An Interactive Multimedia Course of Study, Capter2: Graphing Distributions, Section: Box plot, URL: ns/boxplots.html, Developed by Rice University (Lead Developer), University of Houston Clear Lake, and Tufts University, as of 8 September [14] Watanabe, S., Shirahama, N., Tsukamoto, H., Matsumoto, Y., Nakagawa, M., Miyamoto, K., Naka-ya, N., Tomita, M. and Mori Y.: Distribution of the Subjective Evaluation based on Visual Analog Scale and Likert Scale, Journal of Biomedical Fuzzy Systems Association, Vol.17, No.1, pp.31-38, 2015 (in Japanese). 5

6 Biomedical Soft Computing and Human Sciences, Vol.20, No.2, 2016 Satoshi WATANABE He received his Ph.D. degree in information system from University of East Asia, Shimonoseki, in He joined Shizuoka Sangyo University in He is currently an associate professor at Department of Information Design. His current research interests are informatics, human science and educational technology. He is a member of BMFSA, ASJ, IPSJ, JSET, and so on. Naruki SHIRAHAMA He received his Ph.D. degree in electrical engineering from Kyushu Institute of Technology in He has been working as an associate professor at National Institute of Technology, Kitakyushu College. His present research interests include the application of Subjective Observation Model. He is a member of BMFSA. Masafumi NAKAGAWA He received his Ph.D. degree from Juntendo University. He has been working as a professor at International University of Health and Welfare Hospital. Kazunori MIYAMOTO He received his Ph.D. degree from University of East Asia, Shimonoseki, in He has been working as an associate professor at Kyushu Women s University. He is a member of BMFSA. Naofumi NAKAYA He received his Ph.D. degree from Nihon University in He has been working as a teacher at Tokyo Electronics College. He is a member of BMFSA. Yuji MATSUMOTO He received his Ph.D. degree from Osaka University. He has been working as a professor at Shizuoka Sangyo University. He is a member of BMFSA. Masashi TOMITA He received his M.E. degree from Nagaoka University of Technology. He has been working as an associate professor at Department of Mechanical and Electronic Engineering, Salesian Polytechnic. Hiroyuki TSUKAMOTO He received his B.E. degree from Yokohama National University. He has been working as an associate professor at Shizuoka Sangyo University. He is a senior coach of Japan Sports Association and Japan Volleyball Association. Yukio MORI He received his Ph.D. degree from Tokyo University of Science. He has been working as a professor at Department of Mechanical and Electronic Engineering, Salesian Polytechnic. He is a member of BMFSA. 6