Multidimensional Reliability of Instrument for Measuring Students Attitudes Toward Statistics by Using Semantic Differential Scale

Transcription

1 Amercan Journal of Educatonal Research, 05, Vol. 3, No., Avalable onlne at Scence and Educaton Publshng DOI:0.69/educaton-3--0 Multdmensonal Relablty of Instrument for Measurng Students Atttudes Toward Statstcs by Usng Semantc Dfferental Scale Gaguk Margono * Engneerng Department, State Unversty of Jakarta, Indonesa, Kampus UNJ, Jl. Rawamangun Muka, Rawamangun, Jakarta *Correspondng author: gmargono@unj.ac.d Receved December 4, 04; Revsed January 08, 05; Accepted January 3, 05 Abstract The purpose of ths paper s to compare multdmensonal and undmensonal relablty of nstrument of students atttude towards statstcsby usng semantc dfferental scale. Few researches utlzed multdmensonal relablty measurement. Multdmensonal relablty s calculated by usng Confrmatory Factor Analyss (CFA n Structural Equaton Model (SEM technque. The measurement and calculaton descrbed n ths artcle deal wth nstrument of students atttude towards statstcs.ths nstrument has been tred out to 50 students. It s found that multdmensonal relablty has hgher accuracy compared to the undmensonal one. Perhaps varous formulas applyng multdmensonal relablty would be used n future research. Keywords: multdmensonal relablty, atttudes toward statstcs usng semantc dfferental scale, confrmatory factor analyss Cte Ths Artcle: Gaguk Margono, Multdmensonal Relablty of Instrument for Measurng Students Atttudes Toward Statstcs by Usng Semantc Dfferental Scale. Amercan Journal of Educatonal Research, vol. 3, no. (05: do: 0.69/educaton Introducton Educaton requres relable or trustworthy measurement judgment. Accordng to Naga (99, educatonal and psychologcal measurements nclude several thngs. Frst, t measures respondents latent characterstcs. Second, to measure the latent characterstcs, respondents are gven stmulus through questonnares or approprate measurng nstruments. Thrd, perhaps respondents responses are reflecton of the latent characters. Fourth, the responses are scored and nterpreted adequately []. Then, some questons rse such as: How do the scores reflect the latent characterstcs accurately? Does the nstrument reveal the latent characterstcs (trats properly? Both those questons regard to valdty. In assocate to relablty, we can ask: Are respondents answers realstc to be used for scorng psychologcal attrbutes? Anythng used to measure can be sad as measurement tools. The nstrument has to be valdated before used. Bascally, there are two knds of nstruments; they are test and non test. Test measures maxmum performance and non test calculates atttude (typcal performance. Test wll have true or false answer, whle non test has postve or negatve answer. Accordng to Suryabrata (000 non test measurement needs sentment expresson response that s response whch cannot be judged as true or false answer. Responses here often regarded as true answers relate to each response reason []. Non test does not judge what someone can do; rather t values what someone tends to do. In scentfc research, a good nstrument s acheved through data and t can be better nterpreted through a relable, valuable and objectve process. Accordng to Wreman (986 relablty s consstency of an nstrument to calculate somethng measured [3]. Relablty ndcates how far results of measurement of an nstrument can be trusted. Therefore, relablty s an ndex for ndcatng f an nstrument s valuable and belevable. An nstrument can be stated as a relable nstrument when t measures same symptoms repeatedly and results obtaned are relatvely stable or consstent. Generally, there are three major categores of relablty measurement, they are: ( stablty type (e.g. retest, parallel tems, and alternatve forms, ( homogenety or nternal consstency type (e.g. splt half, Kuder-Rchardson, Cronbach's alpha, theta and omega, and (3 equvalent type (e.g. parallel tems n alternatve forms and nter-rater relablty. In dong ths, an nstrument s gven to one group of subjects once and the relablty estmaton s calculated n certan way. Ths once type applcaton measurement approach generates nformaton about statement consstency of same aspects or reflect statements homogenety. The hgher relablty coeffcent of an nstrument, the closer observed scores to the real scores. So the observed scores can be used as substtuton of real scores. Coeffcent s not the only component n decdng level of relablty. Level of relablty s acqured through calculaton. It also nfluenced by standard of dscplne nvolves n the measurement. Errors decrease when usng a hgh relablty coeffcent nstrument.

2 50 Amercan Journal of Educatonal Research Commonly, affectve characterstcs measurement provdes lower relablty coeffcent than cogntve measurement. It s caused by less stable score of affectve characterstcs. Accordng to Gable (986 relablty coeffcent of cogntve nstrument s usually about 0.90 or more [4], whereas affectve nstrument relablty coeffcent s less than Relablty coeffcent of level 0.70 or more usually can be accepted as a good relablty (Ltwn, 995 [5]. However, Naga (99 says that an adequate relablty coeffcent should be above 0.75 []. Psychologcal research measurement always apples valdty and relablty test. But n psychometrcs, experts stll argue about relablty coeffcent also nter-rater relablty formula. It s caused by some reasons: frst, many competence researchers gve less correct relablty of ther measurements result(thompson, 994 [6]. Second, some researchers use relablty coeffcents monotonously wthout consderng assumptons underles the coeffcent. The researchers do not acknowledge well the use of alpha coeffcents whch requre hard completon assumptons. For ths case, f assumptons do not sut the requrement so alpha coeffcent estmates the lowest lmt pont. Many researchers use alpha coeffcent to estmate relablty. The great range of Cronbach s alpha coeffcent use caused by some factors: computatonal technque used n processng data to get relablty coeffcent s relatvely easy. It only requres total score varance, and samplng dstrbuton s already provded so t s possble to decde true ntervals of populaton (Feld et al., 987 [7]. Thrd, the problem deals wth assumptons n estmatng relablty. Emprcal measurement requres parallelsm. However, tau-equvalent aspect s more complex requrement for measurement. Ths statement supported by Kamata et al (003 who found that assumpton of equalty, test components dscrmnaton, and undmensonalty measurement are relatvely dffcult to be acheved [8]. If tau-equvalent assumpton cannot be obtaned essentally so alpha coeffcent produces very small relablty pont (value. The coeffcent value les under estmaton. Fourth, the man dscourse ssue n measurement s problem when applyng undmensonal measurement. Undmensonalty s an mportant aspect n estmatng relablty. Undmenson psychologcal measurement result s very dffcult to be reached, partcularly n personalty doman context. Ths doman contans broad trats varances area. Socan (000 wrtes that multdmensonal factor analyss studes are conducted more than the undmensonal one [9]. Assumpton problem s not a major ssue n settng nternal consstency models. But t becomes the most chosen topc n relablty study. Research done by Vehkahlat (000 found that unrealstc assumpton n pure classcal theory s genune undmensonal score. Practcally, ths condton s hard to prove [0]. So, study of multdmensonal measurement comes as a soluton for ths. In addton, many cases dscovered that there s ntertem correlaton n the dmenson. Sometmes the correlaton s greater than tem correlaton n test. Educaton researchers use undmensonal assumpton measurement. Ths measurement has concept that there s only one factor of ablty, personalty, affectve and atttude whch measured by one measurement nstrument. But many research showed that undmensonal assumpton s dffcult to be ganed. Because some new factors also dscovered when dong the measurement. In other words, psychologcal nstrument whch often used by researchers tends to be multdmensonal. Multdmensonal relablty measurement s mportant for on some reasons. Accordng to Wdharso (009 the reasons are: frst, generally, characterstcs of psychologcal construct are multdmensonal. Second, psychologcal nstrument nvolves some aspects whch usually started wth tems generated from some theoretcal aspects. The tems tend to be multdmensonal. The thrd reason s the amount of tems n the nstrument. Too many tems can add errors varants potental n tems. It may create new dmensons. Amounts of tems and scale form nfluence respondent atttude towards tems. Ths nfluence wll persuade ther response to the nstrument []. Fourth reason s tem wrtng technques. Spector and colleagues (997 found that tem wrtng technque of two way drecton response; postve (favorable and negatve (unfavorable response may create new measurement dmenson []. In fact, psychologcal scale uses tem wrtng technque whch has dfferent drecton n collectng data. Ffth reason relates to dfferent measurement unts. Psychologcal measurement s lkely to have dfferent measurng unts between one tem wth other tems. It has dfferent capablty measured as of measurement construct. Ths condton wll cause multdmensonal result. Based on the descrptons above t can be concluded that psychologcal measurement tends to be multdmensonal measurement rather than one-dmensonal n both measure cogntve or non cogntve construct. It s suggested that psychometrcs measurement nvolves multdmensonal model analyss technque. McDonald(98 formulates are lablty coeffcent whch namely McDonald composte score relablty coeffcents or omega [3]. The relablty coeffcent based on confrmatory factor analyss whch s part of SEM modelng menu. Ths McDonald composte score relablty explans sze of ndcators proporton n explanng measured construct. Formula for obtanng construct relablty coeffcents s as follows: ω λ = [3] λ + λ Descrptons: ω = McDonald composte relablty (omega λ = Factor loadng of standardzed ndcators to- Accordng to the Latan (0 Structural Equaton Modelng (SEM s a second-generaton multvarate analyss technque that combnes factor analyss and path analyss. Ths technque allows researchers to smultaneously test and estmate relatonshp between exogenous and endogenous multple varables wth many ndcators [4]. In 970s Joreskog research dscovered statstcal theory of lnear structural analyss whch s better known as structural equaton modelng or SEM. Ths modelng uses analyss of covarance structure. So ths approach sometmes called as covarant structure model (CSM. The model ncludes mmeasurable varables called latent

3 Amercan Journal of Educatonal Research 5 constructs whch created by a set of measured varables, namely measured construct. Measurement error whch reflects scores relablty of measurement s seen as a unque construct. It becomes an mportant part of SEM analyss. The error measurement becomes the advantage of SEM analyss compared to other analyss technques (Capraro et al., 00. SEM can estmate error varance of measurement outcome scores that actually estmate relablty [5]. Accordng to Geffen and colleagues (00, SEM s a multvarate statstcal technque that combnes multple regressons whch dentfy relatonshps between constructs and factor analyss [6]. SEM recognzes mmeasurable concept through some manfest ndcators whch both work smultaneously. SEM has some advantages compared to other analyss technques. In studyng relatonshp among varables, SEM automatcally reduces measurement error effect. Capraro et al., (00 says that ndependent varable nfluence towards dependent varable persuaded by attenuaton effect [5]. The value of ths effect s not over the range of relablty coeffcent of test score. Frst approach for ths stuaton s attenuaton correlaton correcton whch caused by measurement error. Second approach s structural equaton modelng n confrmatory factor analyss context. Lee and Song (00 say that SEM s one approach to confrm measurement model [7]. SEM measurement model lnks latent constructs to emprcal construct. Emprcal constructs are expressed by combnaton of latent constructs. SEM may be used n generalzablty theory analyss and tem response theory. SEM s also able to compare measurement models and facltates nvestgaton of model accuracy. Sub model of SEM s factor analyss. Factor analyss s useful for detectng measurement nstrument dmenson. Ths technque ntroduced by Spearman relates to ntellgence factors explorng. SEM also dentfes construct relablty that appears through loadng tems pont produced. Construct relablty s counted through SEM uses ths formula: λ CR = λ + δ Descrptons: CR =Construct relablty λ =Factor loadng of standardzed ndcators to- δ =Standard error of measurement Ths constructs relablty gves same result wth McDonald composte score relablty (omega because δ = λ. The followng rule of multdmensonal relablty coeffcent s construct relablty coeffcents developed by Hancock and Mueller (000 [8]. It shows how well nstrument ndcators can reflect construct whch s beng measured. Ths coeffcent s a modfcaton of McDonald construct relablty coeffcent whch cannot accommodate dfferent weghts of nterdmensons. The modfcaton result s called weghted construct relablty coeffcents as follows: p l ( l Ω w = [8] p l + ( l Descrptons: Ω w = weghted construct relablty (maxmum l = Coeffcent of the -th standardzed ndcator The relablty coeffcent can be nterpreted as square correlaton of dmenson and optmal lnear compostes score. Some experts call t as maxmum relablty. Research done by Wdharso and Mardap (00 expressed that multdmensonal model for relablty coeffcent has greater measurement accuracy compared to undmensonal relablty [9]. For that reason, researcher only focuses on nternal consstent coeffcent lkes α for undmensnal relablty and ω, CR dan Ω w for multdmensonal relablty n ths research. Some questons appear dealng wth the explanaton above such as: What s relablty comparson of multdmesonal and undmensonal of students atttude nstrument towards statstcal by usng semantc dfferental scale? Whch one of both dmensons measurement accurate more for measurng relablty?. Method Ths research uses survey method n developng nstrument uses responses approach. Ths research was carred out at Engneerng Technque Educaton of Engneerng Technque Educaton Program, Engneerng Faculty, State Unverstyof Jakarta. The target populaton was all students of UNJ and the research populaton was all post graduate students of State Unverstyof Jakarta. The sample of the research was Engneerng Technque Educaton students who passed Statstcts course. Ths research used smple random samplng. Research nstrument (questonnares were gven to 60 students, however only 50 students returned the questonnares. Scale s a set of grades or numbers that gven to subject, object, or behavor for quantfcaton and qualty measurement purpose. Scale s used for measurng atttudes, values, nterest, motvaton, and so forth. These elements relate to psychologcal attrbutes (usually affectve area. For example, we can use scale for measurng someone s atttude towards statstcs. Semantc dfferental scale s an nstrument whch s used n evaluatng a stmulant concept on a set of seventh steps bpolar scale from a start pont to the end pont n a unty complaton (Sevlla et al., 993. Pars of adjectve words are usually separated by seventh response categores whch are same unts along the antonym of the words contnuum. The contnuum drecton s usually changed randomly [0]. Semantc dfferental scale here s a set of adjectves words whch refer to stmulant characterstc provded to respondents. If the adjectve words have great factor weghted so t needs a complex analyss called factor analyss. Semantc dfferental scale develops a way n measurng meanng of words whch called Semantc dfferental technque. Meanng s a concept n semantc

4 5 Amercan Journal of Educatonal Research whch s multdmensonal. Ths technque can be used as psychologcal measurement n many aspects such as personalty, atttude, or communcaton. Besdes that, ths technque has specal and unque characterstcs f compared to other methods. One of the unqueness s n the way respondents respond to tems. Respondents are not asked to gve agree or dsagree responses. But they are asked to grade weght of stmulant through adjectve words on each contnuum n the scale. Semantc dfferental scale can be classfed nto three dmensons that are evaluaton (E, potental (P and actvty (A. Evaluaton dmenson s lke good or bad, useful or useless, honest or dshonest, clean or drty, advantage or dsadvantage and so forth. Potental dmenson s such as bg or small, strong or weak, heavy or lght. Actvty dmenson can be actve or passve, quck or slow, and hot or cold. These three dmensons can measure three atttude dmensons that are: (a respondents evaluaton about measured object or concept, (b respondents percepton about object or concept potental, and (c respondents percepton about object actvty. Accordng to Hese (999 evaluaton dmensons nclude nce or awful, good or bad, sweet or sour, dan helpful or unhelpful; potental dmenson s lke bg or lttle, powerful or powerless, strong or weak, and deep or swallow; also actvty dmenson s such as fast or slow, alve or dead, nosy or quet, and young or old []. Isaac dan Mchael (985 descrbe statstcs measurement concept nto: ( evaluaton (E has 5 tems, ( potental (P has 5 tems, and (3 actvty (A has 5 tems []. The target varable n ths research s students atttude towards statstcs. It means someone s tendency towards statstcs wth all hs/her evaluaton, potental, and actvty. The response of ths research s typcal performance responses. It s expected that respondents respond about habt or what they thnk a person usually does or feels when experencng somethng. Ths way of respond s also called expresson of sentment.the expresson s a response whch cannot be judged as a true or false response. All responses are true based on ts reason. Dealng wth ths characterstcs response, the nstrument has certan answer opton range. Each tem has 7 answer choces wth range grade to 7. Respondents have 5 to 0 mnutes to answer. The answer drecton tendency s postve to negatve atttude towards Statstcs. 3. Results and Dscusson Frst Try out The nstrument of atttude towards statstcs has 5 tems, conssts of 5 tems for evaluaton dmenson, 5 tems for potental dmenson, and 5 tems for actvty dmenson. Alpha Cronbach nternal consstency relablty s reached from SPSS program that s McDonald composte sore relablty uses structural equaton modelng (SEM ganed ths result: λ = dan λ = ; so (9.080 ω = = ( (8.967 Construct relablty obtaned the same result as follows: λ = and δ = ; so (9.080 CR = = ( (8.980 Weghted construct relablty used SEM and produced p l ths result: = 5.03, so t can be counted as: ( l 5.03 Ω w = = Second Try Out The nstrument has 5 tems conssts of 5 tems for evaluaton dmenson, 5 tems for potental dmenson, and 5 tems for actvty dmenson. Alpha Cronbach nternal consstency relablty processed through SPSS program that s 0.9. McDonald composte score relablty uses structural equaton modellng (SEM got ths result: λ = 0.90 and λ = 7.47; so (0.90 ω = = ( (7.47 Construct relablty obtaned same result as follows: λ = 0.90 and δ = 7.60 so (0.90 CR = = ( (7.60 Weghted relablty used SEM and produced ths result: p l = , so ths can be calculated as ( l Ω w = = It can be summarzed as the table below: Relablty Coeffcent Table. Summary of Relablty Coeffcent Undmensonal Multdmensonal α ω CR Ω w Try Out Try Out The value (grade of alpha Cronbach coeffcent whch was acheved s smaller f compared to construct relablty, McDonald composte score relablty, and maxmum relablty. The dfference s 0.03 to 0,060. However, does the dfference express accuracy? There s no agreement among nter psychometrcs experts about ths. Because the mportance role of usng accurate relablty measurement nstrument, Indonesa researchers should try to use ths tool correctly and adequately.

5 Amercan Journal of Educatonal Research 53 Most researchers from lecturers or post graduate students do not know the formula to count relablty coeffcent usng SEM. Ths s the tme to ntroduce and use the formula. The reasons for that are we already knew the rules and most of psychologcal, personalty, educaton, and socal construct s multdmensonal dmenson. So all the researchers should try to develop and study more about relablty coeffcents. Interpretaton of relablty coeffcent s evaluaton of test score cautousness. It s not only about relablty tself. Two thngs that should be consdered when nterpretng level of relablty coeffcent, that s: ( coeffcent relablty of certan group subjects and stuaton wll not be the same as other group, and ( relablty coeffcent only ndcates score nconsstency of measurement result. It s not for statng the causes of the nconsstency. Educaton measurement s complcated. Many journals artcles dscuss about measurement whch should gve vald, relable, and accurate result. It s not easy snce t nvolves mathematcs knowledge. We cannot understand varous educaton measurement journals f we do not master hgh level and complcated mathematcs applcatons. We left behnd so far n educaton measurement. Not many educaton experts understand content of educaton measurement journals wth great level of mathematcs applcaton. Because of that, mprovng educaton measurement program s crucal. The frst effort for such program s change our percepton about mathematcs. Some educators stll thnk that educaton knowledge does not need mathematcs much. Mathematcs s only part of scence and techncal dscplnes, not for educaton man dscplne. However, today the percepton should be changed. Educators need to realze that not all study or dscplnes use mathematcs applcaton, but some are really need t, lke the example above whch apples multvarate statstcs that requres great level mathematcs applcaton. 4. Concluson It can be concluded that multdmensonal relablty coeffcent work more correctly to estmate relablty than undmensonal relablty. There some suggestons for that: frst, ths nstrument estmaton need to be carred out n an advance study. Second, ths research used scale of 7, other scales can also be appled, such as Lkert scale, dchotomy scale, Thurstone scale, and so forth. Thrd, ths nstrument needs to be examned n larger populaton and wder settng whch nvolve some provnces, varous levels of schools and colleges. Fourth, wde use of multdmensonal relablty analyss around students or researchers study may be valuable for mprovng accurate result of research. References [] Naga, Dal S., Teor sekor, Gundarma Press, Jakarta, 99. [] Suryabrata, S., Pengembangan alat ukur pskologs, And Offset, Yogyakarta, 000. [3] Wersma, W., Research methods n educaton: An ntroducton, Allyn and Bacon, Inc., Boston, 986. [4] Gable, R. K., Instrument Development n the Affectve Doman, Kluwer Njhoff Publshng, Amsterdam, 986. [5] Ltwn, M. S., How to Measure survey relablty and valdty, Sage Publcatons, London, 995. [6] Thompson, B., Gudelnes for author, Educatonal andpsychologcal measurement, 54, [7] Feld, I. S., Woodruff, D. J., and Salh, F. A., Statstcal Inference for coeffcent alpha, appled psychologcal measurement, II, [8] Kamata, Turhan, A., A., and Darandar. E., Estmatng relablty for multdmensonal composte scales scores, Paper presented n annual meetng of Amercan Educatonal Research Assocaton d Chcago. Aprl 003. [9] Socan, G., Assessment of relablty when test tems are not essentally t-equvalent, In Development n survey methodology, Anuska Felgoj and Andrej Mrvar (Eds., FDV, Ljubljana, 000. [0] Vehkalaht, K., Relablty of Measurement scales tarkkonnen s general method supersedes cronbach s alpha, Academc Dssertaton, Unversty of Helsnk, Fnland [] Wdharso, W., Koefsen Relabltas pada pengukuran keprbadan yang bersfat multdmens, Pskobuana, (, [] Spector, P., Brannck, P., and Chen, P. When two factors don t reflect two constructs: how tem characterstcs can produce artfctual factors, Journal of management, 3 (5, [3] McDonald, R. P., The Dmensonalty of test and tems, Brtsh journal of mathematcal and statstcal psychology, 34: [4] Latan., H., Structural equaton modelng konsep dan aplkas menggunakan program lsrel 8.80, Alfabeta, Bandung, 0. [5] Capraro, M. M., Capraro, R. M., and Herson, R. K., Measurement error of score on the mathematcs anxety ratng scale across sudes, Educatonal and psychologcal measurement, 6: [6] Geffen, D., Straub, D. W., and Boudreau. M. D., Structural equaton modelng and regresson: Gudelnes for research practce, Communcatons of AIS, 4, Artcle [7] Lee, S. Y., and Song, X. Y., Hyphotess testng and model comparson n two-level structural equaton model, Multvarate behavoral research. Volume 36, Issue 4: [8] Hancock, G. R., and Mueller, R. O., Rethnkng construct relablty wthn latent varable systems, In Stuctural equaton modelng: Present and future, R. Cudek, S. H. C. dutot, dan D. F. Sorbom (Eds., Scentfc Software Internatonal, Chcago, 000. [9] Wdharso, W., and Mardap, D., Komparas ketepatan estmas koefsen relabltas teor skor murn klask, Jurnal peneltandan evaluas penddkan, 4 (: [0] Hese, D. R., The semantc dfferental and atttude research, (accessed 0 November 03. [] Sevlla, C. G., Ochave,J. APunsalan, T. G., Regala, B. P.,dan Urarte, G. G., Pengantar metode peneltan, translated by Almuddn Tuwu, Penerbt Unverstas Indonesa, Jakarta: 993. [] Isaac, S., and Mchael. W. B., Handbook n Rresearch and evaluaton: For educaton and the behavoralsscences, Edts Publshers, Calforna, 985.