DOCTORAL THESIS. Using Multiple Performance Parameters in Testing Small Graphical Symbols. Dominic Paul T. Piamonte.

Transcription

1 2000:02 DOCTORAL THESIS Using Multiple Performnce Prmeters in Testing Smll Grphicl Symbols Dominic Pul T. Pimonte Doctorl thesis Institutionen för Arbetsvetenskp Avdelningen för Industriell ergonomi 2000:02 ISSN: ISRN: LTU-DT--00/02--SE

2 2

3 ABSTRACT Grphicl symbols hve become minsty in interfces of most modern-dy industril mchines nd personl tools with their incresed computing power nd ever-shrinking sizes. Their use hs trnscended ntionl boundries nd is seen ll over the world. However, only ltely hve studies strted to gther momentum, ttempting to nlyse how they re relly understood by people of different bckgrounds nd needs, nd how helpful cn they relly be in mking mn-mchine interctions better. Amidst this sitution, this doctorl study ws conceived. The min objective of the study ws to nlyse the understnding of smll grphicl symbols (pictogrms nd icons) mong different potentil user groups round the world. Different test indices or prmeters were used to exmine how symbol understnding would differ or be similr cross different study groups. An initil study done in the Philippines nd Sweden using three sets of twenty-three videophone symbols designed in Jpn, USA nd Englnd preceded the min study. The min study utilised three sets of videophone symbols nd were bsed on studies done by the Humn Fctors Technicl Committee (HFTC) of the Europen Telecommuniction Stndrds Institute (ETSI). Subjects from Indonesi, Mlysi, Philippines, Thilnd, Sri Lnk, s well s subjects from Finlnd, Sweden nd USA prticipted in the study. Bsed on the results nd lernings from the initil study, s well s considering the recommendtions of the ETSI studies, more mesurement prmeters were used. Thus, four tests were used, nmely: ) spontneous identifiction or free recll, b) the cued response, c) semntic rting scles, nd d) preference tests. Confidence judgements complemented the subjects nswers for the first two prts. These four tests yielded 10 test prmeters tht were used in compring symbol understnding of the different country groups. These prmeters were: (1-4) spontneous identifiction nd cued response hit rtes nd certinty rtings (confidence judgements); 5) misses; 6) flse lrms or confusions, 7) missing vlues; 8) individul symbol preference, 9) symbol set preference; nd 10) semntic rting scles. Results from the spontneous identifiction tests in ll countries reveled very poor identifiction of most symbols in contrst to the cued response test results. The first test simultes the users initil encounters with the symbols. Brely recognising wht the symbols ment implied the need to either redesign the symbols or to ensure dequte opportunities for fmilirising nd educting the prospective users with the new symbols. The subjective certinty scores helped in studying the level of confidence of the nswers by the subjects. Furthermore, the studies reveled tht symbols could be esily recognised (high hit rtes) but lso confused s representing nother (wrong) function t the sme time. These were the "flse lrms," instnces when the symbols were chosen under the wrong referent contexts. The missing vlues were likewise importnt since they indicted situtions when respondents either did not know the nswer or thought tht none mong the symbols were comprehensible or representtive of desired function. The preference tests pertined to esthetics of the symbols individully nd s set. In turn, the SDT scores reveled tht symbols could hve different connottive menings in reltion to the functions they were intended to represent. Between countries, Asin subjects performed comprbly well with the Europen nd Americn subjects, but usully t the expense of more errors nd confusions. Tests with the elderly reveled other importnt issues in symbols evlution. For exmple, they filed to perform the free recll nd semntic tests climing they were either too difficult or too complicted, strongly suggesting the need to hve simpler nd more prcticl tests for the elderly. In ll the results implied the grvity of the problem tht cn occur if grphicl symbols re introduced rbitrrily without considering nd involving the prospective users t the erliest design stges possible. 3

4 4

5 TABLE OF CONTENTS ABSTRACT 3 PREFACE 9 ACKNOWLEDGEMENTS 11 PAPERS PRODUCED FROM THE STUDIES 13 SUMMARY OF PAPER 1 15 SUMMARY OF PAPER 2 17 SUMMARY OF PAPER 3 19 SUMMARY OF PAPER 4 21 SUMMARY OF PAPER 5 23 SUMMARY OF PAPER 6 25 SUMMARY OF PAPER INTRODUCTION FROM SIGNS AND SYMBOLS TO ICONS: A BACKGROUND ON GRAPHICAL SYMBOLS DEFINITION OF TERMS SOME ASPECTS OF COGNITIVE ERGONOMICS OF GRAPHICAL SYMBOLS Visul imge processing The role of context in symbol mening The sign-signl-referent reltionship Symbol clssifictions THE USE OF GRAPHICAL SYMBOLS THROUGHOUT THE TIMES From pintings to printing press Modern technology, globlistion nd the return of the grphicl symbols 35 5

6 2 CURRENT PROBLEMS AND ISSUES RESTRICTIONS ON THE ADVANTAGES OF USING GRAPHICAL SYMBOLS STANDARDISATION PROBLEMS SYMBOL EVALUATION ISSUES Current test methods Use of severl test prmeters The MIA - Multiple Index Approch The semntic differentil test Prospective users s subjects Culture nd symbol evlution 42 3 RATIONALE AND OBJECTIVES OF THE PRESENT STUDY 44 4 HYPOTHESES 45 5 METHODOLOGY INITIAL STUDY: PHILIPPINE-SWEDISH STUDY Respondents Mterils nd equipment Procedure MAIN STUDY: SOUTHEAST ASIA, EUROPE AND USA (PAPERS 2-7) Respondents Mterils nd equipment Procedure Experimentl design 52 6 RESULTS AND DISCUSSION INITIAL STUDY: PHILIPPINE-SWEDISH STUDY Min effects nd interctions Certinty rtings Semntic Differentil Scles MAIN STUDY: SOUTHEAST ASIA, SWEDEN, USA, FINLAND (PAPERS 2-7) Exposure to computer-relted devices Attitudes towrds technology Hit rtes Confusions nd missing vlues Combining the objective performnce prmeters 73 6

7 6.2.6 Subjective certinties Symbol nd set preferences Semntic Differentil Tests The Finnish Elderly (Pper 5) Generl discussion 83 7 CONCLUSIONS AND RECOMMENDATIONS 86 8 REFERENCES 88 PAPERS PRODUCED FROM THE STUDIES 93 APPENDICES 7

8 8

9 PREFACE The term technology generlly pertins to the processes humn beings use to design nd utilise tools nd mchines to increse their control nd understnding of the mteril environment. The word technology comes from two Greek words: tekhne, which refers to n rt or crft, nd logi, mening n re of study. Thus, technology literlly mens the study, or science, of crfting (Encrt, 1997). It is often stted tht technology is n essentil condition of dvnced, industril civilistion, tht technologicl development is supposed to led to the improvement of qulity of humn life - to help us do things better, fster, sfer nd more efficiently. However, it is lso climed tht the rte of technologicl chnge hs developed its own momentum in recent centuries. Innovtions now seem to pper t rte tht increses geometriclly, without respect to geogrphicl limits or politicl systems. These innovtions tend to trnsform trditionl culturl systems, frequently with unexpected socil consequences. Specificlly, the tempttion to forget the humn being s the centre nd reson for such development hs indeed become problem. Thus, technology cn be lso conceived s "both cretive nd destructive process, " (Encrt, 1997). Ergonomics, despite its reltively young ge of existence s n ccepted nd essentil scientific discipline, hs been t the forefront of putting mn where he belongs midst ll the technology - the mker nd controller of his work. Ergonomics involves the study of the humn being s physicl nd psychosocil trits, cpcities, nd limittions. It is n pplied science utilising such knowledge to mtch the person to his/her tsks nd tools. Thus, in its brodest sense, ergonomics is indeed "the science nd prctice of fitting the tsk (technology s tools, equipment, nd interfces) to the humn being" (Kroemer, 1997). This doctorl thesis is just humble exmple of striving ergonomist s ttempt to contribute to little knowledge nd insight on the ergonomics of mn-mchine interfces. It is n re reflective of the current rpid pce of technologicl development. Indeed, the use of grphicl symbols personifies the current economic nd technologicl globlistion - they re found everywhere nd re used by more nd more people. However, question still remins - re they lwys understood the wy they re intended to be understood? This pper, in essence, ttempts to nswer this question. 9

10 10

11 ACKNOWLEDGEMENTS It is quite esy to ly personl clim to n invention or discovery becuse of one s hrdship nd persevernce. It is likewise esy to contend tht knowledge cn be obtined by oneself without nyone s help, if he or she perseveres hrd enough in studying nd doing reserch lone. However, I sincerely believe tht person s knowledge nd skills encompss collection of the experiences, knowledge, nd expertise derived from countless people person hs come cross during his or her lifetime. Thus, one cnnot be bostful or full of pride when knowledge is gined, honed, nd brodened. Insted, one should become ll the more pprecitive nd indebted to ll the people behind such pursuit of knowledge. In my cse, there re countless of them. Below re those to whom I m indebted throughout my reserch work. This doctorl work is s much theirs s it is mine. My thnks nd pryers to: To Prof. Kjell Ohlsson, my mible nd rock-solid supporter nd supervisor, lwys with time to shre nd knowledge to imprt To Dr. John "Abey" Abeyseker, the person who relly strted this journey for me wy bck during the M.Sc. dys - the one who hs never cesed to hve fith nd belief in me nd in wht we hve done To Prof. Houshng Shhnvz, my revered "sensei", lwys source of wisdom nd guidnce To Dr. Mrtin Böcker, who never cesed to lend helping hnd wherever he my be To My Collegues nd Clssmtes, from M.Sc. dys to Ph.D. dys, the shred hrdships, triumphs, ters nd lughter To All the Subjects in the studies, for their willingness to prticipte nd shre their time nd effort To Ingrid nd Anit, ever present to ssist nd mke every dy s work not just bit esier but lso more rewrding To My Prents, for their boundless fith nd ever-powerful pryers To My Fmily, we ll did this together And, to The Lord Almighty, my infinite source of inner strength nd inspirtion MARAMING SALAMAT PO! TACK SÅ MYCKET! 11

12 12

13 PAPERS PRODUCED FROM THE STUDIES This doctorl thesis is bsed on the following refereed journl nd conference ppers s well s some unpublished dt: 1. Pimonte, D.P.T., Abeyseker, J.D.A. & Ohlsson, K. (1996). A user-bsed evlution of telecommuniction icons: some cross-culturl issues. In L.K. Yong, L., Hermn, Y.K. Leung & J. Moyes (Eds.), Humn Fctors of IT: Enhncing Productivity nd Qulity of Life, Proceedings of the First Asi-Pcific Conference on Computer-Humn Interfce (pp ). Singpore: Informtion Technology Institute. 2. Pimonte, D.P.T., Ohlsson, K., nd Abeyseker, J.D.A. (2000). Evluting public grphic symbols using multiple test prmeters. (for publiction) Interntionl Journl on Occuptionl Ergonomics. 3. Pimonte, D.P.T. (1999). The relevnce of error nlysis in grphicl symbols evlution. Interntionl Journl of Occuptionl Sfety nd Ergonomics, 5(4), Pimonte, D.P.T., Abeyseker, J.D.A. & Ohlsson, K. (1999). Testing videophone grphicl symbols in Southest Asi. In H.J. Bullinger & J. Ziegler (Eds.), Humn-Computer Interction: Ergonomics nd User Interfces: Vol. 1 (pp ). London: Lwrence Erblum Assocites. 5. Pimonte, D.P.T., Ohlsson, K., nd Abeyseker, J.D.A. (1999). An empiricl evlution of videophone symbols: n interntionl study. In H.J. Bullinger & J. Ziegler (Eds.), Humn-Computer Interction: Ergonomics nd User Interfces: Vol. 1 (pp ). London: Lwrence Erblum Assocites. 6. Pimonte, D.P.T., Abeyseker, J.D.A., nd Ohlsson, K. (2000). Understnding smll grphicl symbols: cross-culturl study (ccepted for publiction). Interntionl Journl of Industril Ergonomics. Amsterdm: Elsevier Science Publishers B.V. 7. Pimonte, D.P.T., Ohlsson, K., nd Abeyseker, J.D.A. (2000). On the merits of using multiple indices in evluting smll grphicl symbols (ccepted for pper presenttion). In Interntionl Ergonomics Assocition Triennil Conference IEA 2000, Sn Diego, Cliforni. 13

14 14

15 SUMMARY OF PAPER 1 The first pper ws study imed to evlute grphicl symbols (icons) designed in Jpn nd Europe (industrilised countries) with subjects coming from the Philippines (n industrilly developing country). The 23 icon referents nd the 69 icons used by Tudor (1994) were selected. Sixty-two of the icons were designed in Englnd. The remining 7 icons were designed in Jpn. The method ws the questionnire type of testing. Ech questionnire hd two prts: recognition/subjective certinty tests, nd semntic differentil. Questionnires were produced per icon fmily with the icons nd their referents presented in rndom orders. The semntic differentil ws composed of 6 bipolr djectives in 7 point scles lso rrnged in rndom orders for ech of the icons or referents. After presenting the study s objectives nd explining the icons, the subjects first underwent the recognition nd certinty tests. Icons nd referents were rndomly presented nd to be mtched correctly. After one week, the semntic differentil testing ws performed. The icons were gin presented. The sme tsks were done for the three icon fmilies. Thus, ech subject performed 3 recognition/subjective certinty tests, nd 3 semntic differentil tests, one for ech icon fmily. The results showed tht for the 23 referents represented by 23 icon fmilies, only 5 of the 23 referents hd men recognition percentge of t lest 60 percent. Further, the icons with high men recognition percentges lso exhibited high men rting scores for the certinty tests. Rnk order correltions were lso t lest for these icons. Icons with low recognition mens hd low certinty levels nd semntic scores. For the SD tests, results showed tht correltions were high between the semntic scles nd the percent men of correct recognition regrdless of icon fmilies. Compred to Americn subjects used in nother study, the Filipino subjects fred better with the Jpnese-designed icons. Such difference my be reflection of Philippine socil condition where mjority of current computerised nd electronic pplinces in the Philippines is imported from Jpn. Americns on the other hnd re obviously more kin to be exposed to products they hve mde. Although USA still rnks s the mjor trding prtner of the Philippines followed by Jpn, the ltter rnks first in terms of the computer nd telecom products mrketed nd used loclly. Icons re lso commonly found in these products. This renders the view tht culturl fmilirity nd mening of icons cn ffect their usbility when pplied s interfce tools in new or modern technologies. The SD results showed tht SD testing could be vlid index of how messges of symbols like the icons re perceived nd understood. It is simple, inexpensive but relible. Admittedly though, there re limittions in this pilot study. A distinct group of potentil users (educted nd computer literte) ws used s subjects. Future studies will need to utilise vrying groups of potentil users (students, professionls, experts, novices, etc.) cross different cultures of Asi. 15

16 16

17 SUMMARY OF PAPER 2 The next five ppers, including this one used three sets of grphicl symbols representing seven videophone functions or referents designed nd tested in the West (Europe). Here, the respondents were from Indonesi, Mlysi, Philippines nd Thilnd. More tests were likewise used nd were bsed on the ETSI studies by Böcker (1993). In the spontneous identifiction or free recll test (Prt One), ech pge contined one set of the seven videophone symbols, subjects would write wht videophone function they thought ws represented by ech symbol. They were lso sked to rte the level of certinty for ech of their nswers using seven-point rting scle. In the cued response test (Prt Two), the subjects first red referent nd its description then they chose one symbol from set of seven symbols which best represented the referent in question. They were gin sked to rte their certinties for their nswers using the 7-point rting scles. For the lst prt, the subjects would choose seven symbols to represent the seven referents nd one symbol set they preferred ech subject group to minimise order nd lerning effects. In ll, the test prmeters studied were hit rtes (men percentges of correct identifiction/recognition in Prts One nd Two), subjective certinty rtings, missing vlues, confusions, nd symbol nd symbol set preferences. Overll, Set 1 s symbols (recommended ETSI symbols) hd the highest hit rtes compred to Sets 2 nd 3 in lmost ll of the 7 referents or functions tested. Tking between tests s nother min effect, the hit rtes in the cued response tests (Prt Two) were significntly higher thn in the spontneous identifiction tests (Prt One) for ll the 7 referents. Symbols with high hit rtes exhibited high certinty rtings s well - gin, mostly symbols from Sets 1nd 3. Confusions were instnces when symbol ws ssocited to the wrong referents. The different countries were quite similr in the confusions of the symbols esily identifying fmilir symbols ("cmer" nd "document cmer") nd confusing them with other functions. Further, the symbols of Set 1 were the preferred symbols by t lest 50% of the subjects for most of the referents. Prt One s results showed tht grphic symbols could be quite difficult to understnd or comprehend for the Asin subjects bsed on the very low hit rtes. However, the high improvements in hit rtes in Prt Two showed tht with proper instructions nd cues, Asin subjects would not hve mjor problems recognising grphic symbols. Certinty levels nd missing vlues could help further qulify the levels of hit rtes ttined by the symbols. Flse lrms or confusions were demonstrted to represent the degree of confusing symbol to nother symbol presented t the sme time. In ll, multiple test prmeters were useful in identifying not only the best symbols to use but lso other potentil problems in using symbols mong different countries, which in turn my be helpful in considering needed chnges or improvements. 17

18 18

19 SUMMARY OF PAPER 3 If symbols re intended s prts of interfces of devices for interntionl use or for stndrdistion, different tests re usully needed. The previous study (Pper 2) hd lredy prtilly shown this. This pper would concentrte on the so-clled non-hit prmeters. Two hundred forty university students nd employees nd professionls (127 mles nd 113 femles) from smll nd lrge compnies from five Asin countries prticipted (Indonesi, Mlysi, Philippines, Thilnd nd Sri Lnk). Agin, the stimuli were three sets of symbols representing seven referents or functions of videophone for totl of twenty-one symbols used in the previous pper. Both referents nd symbols were bsed on the ETSI study heded by Böcker (1993). The questionnires were divided into three prts: symbol identifiction/subjective certinty tests, symbol ssocition/subjective certinty tests, nd symbol nd set preferences. When ISO 9186 nd ANSI Z535.3 comprehension levels were considered through the hit rtes, most of the tested videophone symbols performed poorly. Combining country results, only seven of the 21 symbols reched ISO s required comprehension level of t lest 67%. Furthermore, the wrong nswers during the cued response tests were nlysed using confusion mtrices. Symmetric confusions usully suggest visul or conceptul similrities. Asymmetric confusions occur when subjects simply chose the wrong symbol for given referent. Across symbol sets, "selfview" nd "videophone" symbols hd the highest instnces of being mistken s representing other functions. More differences were noted when the countries were compred bsed on symbol confusions. The results lso showed tht confusions could be useful indictors of the suitbility of the symbol nd my even complement the hit rtes. For exmple, the videophone symbols of Sets 1 nd 3 hd very similr hit rtes mong the countries. With regrds to the microphone symbols wherein subjects from the five countries hd similr hit rtes lso, Set 3 s version performed best for Indonesi, Mlysi nd Sri Lnk (lower symbol confusions). Determintion of the hit rtes gives the level of ssocition of the symbol to its intended referent or function. However, when symbols pper simultneously, the hit rtes cnnot revel the dynmics of how ech symbol cn ffect the other s level comprehension nd ssocition to its referent. Confusions re instnces when symbols re selected under the wrong referent contexts. Together with hit rtes, they cn give n ide on symbol s distinctness from other symbols under different referent contexts s well s the different ptterns of their interctions in different groups. Other prmeters lso exist, which were not discussed here but could lso be helpful in evluting grphicl symbols. 19

20 20

21 SUMMARY OF PAPER 4 This pper ws summry of the study on videophone symbols using both objective nd subjective prmeters with respondents from the five Southest Asin countries. It used the sme methods bsed on ETSI studies by Böcker (1993) with some modifictions (i.e. inclusion of the spontneous identifiction tests). Between tests, correct responses in the cued response tests (Prt Two) were significntly higher thn in the spontneous identifiction tests (Prt One), except for the symbols for "microphone". Furthermore, only the symbols (of Sets 1 nd 3) for referents microphone nd videophone grnered hit rtes of t lest 67% in Prt One. Altogether in both tests nd cross ll symbol sets, the highest hit rtes were for "cmer", "microphone" nd "videophone" while the "document cmer", "hndsfree", "selfview nd still picture hd the lowest hit rtes in ll countries. Compring countries, there were significnt differences in hit rtes for symbols for cmer, document cmer, hndsfree. In these symbols, Sri Lnkn subjects generlly hd the lowest hits compred to the other four countries. No significnt differences were noted when subjects were compred bsed on occuption (student vs. employees/professionls). Tbultions mde on the rtes of confusions showed more differences between countries. For exmple, while Thilnd hd the most instnces of confusing Set 1 s symbol for cmer thn the other four countries, it hd the lowest instnces of confusing Set 1 s videophone symbol for other functions. Confusions re very useful indictors of the suitbility of the symbol nd my even complement the hit rtes. For exmple, with regrds to the microphone symbols wherein subjects from the five countries hd similr hit rtes lso, Set 3 s version performed best for Indonesi, Mlysi nd Sri Lnk (lower symbol confusions). Lstly, missing vlues were instnces when some of the subjects gve no response or nswers during the cued response tests. Between countries, Thilnd hd the most instnces of missing vlues mong ll countries. Sri Lnk ws the opposite with no missing vlues in the cued response test. The results in the spontneous identifiction nd cued response tests in testing videophone symbols showed very low rtes of correct identifiction nd even correct ssocition of the symbols cross ll countries. Differences between countries lie mostly in the levels of confusing the symbols from ech other nd their missing vlues. 21

22 22

23 SUMMARY OF PAPER 5 The previous ppers delt with the study on videophone symbols using subjects from five Southest Asin countries. This pper would discuss the prt of the reserch involving U.S. nd Finnish elderly subjects nd compring them with the Asin results. The sme mterils (symbols nd questionnires) nd method used in the previous ppers were used in this study. Further, side from the spontneous identifiction nd cued response tests, symbol nd set preferences were lso exmine. This ment tht the subjects hd to choose symbols nd set they thought best represent the referents individully nd s set, respectively. In ll, the subjects would choose seven symbols to represent the seven referents nd one symbol set they preferred most. The still picture symbols filed to score hits bove 67% in both Prt One nd Prt Two. In contrst, the symbols for cmer, microphone nd videophone hd high recognition rtes regrdless of test type nd set. Regrding preferences, the US subjects preferred five of seven of Set 1 s symbols. These results were similr to the Southest Asin study. One importnt fctor tht leds subjects or potentil users to their choice of symbols is esy recognition. In some prts of Northern Finlnd, videophone-bsed services for the elderly re being tested. This ws one of the mjor resons why the current study ws lso tested mong group of elderly subjects. Compred to the subjects from Southest Asi nd even the USA, the Finnish elderly filed to do Prt One. Regrding Prt Two, nd when compred to younger subjects (from Asi nd the US), the elderly subjects hd much lower hit rtes in ll symbols of the three sets used. The trend of demogrphic evolution shows tht the Europen popultion, in prticulr, is one tht is becoming old. This trend will hve implictions on the socil structure, where respect for n independent living should be tken into considertion. It should tking into ccount the new services nd products just vilble or those under development, such s telelrms nd teleservices to ssist people t home or elsewhere. The results seemed to suggest tht only by conscious nd ctive prticiption of the elderly in designing nd developing modern products would their needs be truly considered nd met. As whole, empiricl tests using multiple indices re importnt to properly evlute the symbols on their usbility especilly cross different user groups. This cn be exemplified by the elderly nd US subject groups who chose symbol sets quite different from the Asins. Hit rte is n importnt prmeter, but must be tested under different user scenrios to determine if the symbols indeed re understood with nd without cues. At the sme time hits rtes lone re not enough. Subjective certinties, flse lrms (confusions), preferences, nd even missing vlues re eqully useful fctors needed to help mke deeper nlyses. 23

24 24

25 SUMMARY OF PAPER 6 This study ws prt of the interntionl project on evluting telecommuniction symbols. It would report on the prts involving Europen (Swedish) nd US (Americn) subjects. The sme test method used in previous ppers ws used in testing the sme three sets of icons nd pictogrms representing seven referents or functions of the videophone. Ech questionnire hd four prts: symbol identifiction tests, cued response tests, semntic differentil rtings, nd symbol nd symbol set preferences. Regrding identifiction nd cued-response tests (Prts One nd Two), Set 1 symbols generlly hd the higher hit rtes in both countries. However, both countries performed poorly in the spontneous identifiction tests (Prt One). For exmple, subjects from both countries mnged to ttin the Orgniztion for Interntionl Stndrdiztion s ISO 3864 (ISO, 1984) minimum correct recognition rte of 66.7% in only 5 of the 21 symbols tested. The poor results thus strongly suggest tht without sufficient lerning opportunities nd lerning ids, symbols my be hrder to understnd by lmost nyone thn previously thought. In ddition, when trying to study possible culturl influences in symbol evlution, levels of correct identifiction re not enough. Compred to the Americn subjects, Swedish subjects tended to give lower certinty rtings especilly during the spontneous identifiction tests. This cn suggest certin subject bis. Confusion mtrices lso helped to show other differences between the symbol sets nd country groupings. Semntic scles hve likewise been shown very useful in studying how symbols re perceived. The Swedish group s whole gve lower men semntic rtings compred to the Americn group. As mentioned bove, the differences in ptterns of rtings suggest tht some subject bis occur between the two groups, which my be culturlly-linked nd cn help in determining which spects of symbol design nd usge my be more helpful (ex. instructions, lerning ids, etc.). Awreness of such subject bis nd their implictions re importnt on how one interprets the test results. Admittedly though, the bove findings need to be nlysed more deeply to be ble to concretely trnsform such differences into prcticl pplictions of symbol design nd testing. 25

26 26

27 SUMMARY OF PAPER 7 This pper ttempted to summrise the results of ll the tests using the ETSIrecommended symbols for 7 videophone functions or referents nd to develop some prcticl recommendtions. The results of the tests in the different countries showed tht the spontneous identifiction test (Prt One) hd significntly lower hit rtes compred with the hit rtes in the cued response test (Prt Two). Prt One is sitution kin to using the technology the first time with miniml knowledge nd instructions. Thus, it ws not surprising tht only fmilir symbols or those tht re concrete representtions of the functions would hve higher hit rtes (ex. microphone, nd cmer), confirming one of the min hypotheses of this study. Further, it ws interesting to note tht symbols could indeed be quite difficult to understnd or comprehend bsed on the very low hit rtes in Prt One. On the other hnd, by using the cued-response tests severl objective prmeters were derived tht helped revel other possible differences between the cndidte symbols nd between subject groups. These cn be summrised s follows. Overll, hit rtes or correct symbol-referent ssocitions re still the most importnt objective prmeter of symbol performnce. However, hit rtes re not the only importnt prmeter, especilly when evluting severl sets of symbols nd using different subject groups. Non-hit prmeters such s flse lrms or confusions nd missing vlues cn be of gret importnce s well. Flse lrms or confusing the symbol s representing the wrong referent is very good guge of studying the how distinct symbol is compred to the other symbols presented t the sme time. This prmeter becomes more useful when used mong subject groups with different culturl bckgrounds (i.e. countries). The predominnt types of confusions (symmetricl nd symmetricl) in ech group could be used to study further how the symbols re understood nd even confused by the subjects. These would then be helpful if to determine if the problems would merit () redesign of the symbols; (b) replcement of the symbols; (c) more explntion or emphsis on how the definition of the functions. The issue would then be whether the recommendtions should focus on symbol re-design or replcements. Another would be on the use of instructionl ids. Missing vlues re lso very importnt since they represent situtions wherein the prospective user plinly lcks the knowledge of which mong the symbols represent the desired function (referent). In ctul user scenrios, they cn be kin to non-use or under-use of the device. If this would be one of the min problems, recommendtions would be more on steps ensuring fmilirity with the device nd its functions. Lstly, the confidence judgements shown in subjective certinties, semntic scles nd preferences were lso helpful in gining insight in subject group differences especilly to certin possible subjective bises tht my be culture-relted. They lso showed tht when intended for diverse user groups, grphicl symbols tht re more concrete nd fmlir hve better chnces of being understood correctly (ssocited to the correct functions). 27

28 28

29 1 INTRODUCTION 1.1 From signs nd symbols to icons: A bckground on grphicl symbols The ubiquitous nture nd pplicbility of the current computer nd informtion technology re quite plpble nowdys. Innumerble tools nd equipment in schools, homes nd workplces hve become computer-bsed. From the lowly flt irons, ovens nd telephones, to different udio-video pplinces nd high- tech desktop nd notebook computers, ll of these hve become equipped with more complex functions. Consequently, one needs to lern how to mnipulte grphic displys nd buttons to vil of the dded functions. In the midst of these complicted mchines nd the humns' ttempt to utilise them, re the grphicl symbols. This introduction is literry discourse on the roles plyed by grphicl symbols in humn-mchine interctions, prticulrly those involving grphicl user interfces. It would discuss grphicl symbols in the relms of cognitive ergonomics nd humn-computer interction. It would strt with some definition of terms followed by discussion on how symbols re probbly processed nd understood. It would then proceed to brief history of the evolution of grphicl symbols - from being mjor prt of dily written lnguges in the pst, to their current role in grphicl user interfces (GUIs) of modern-dy devices. 1.2 Definition of terms The dvent of computer technology nd the subsequent widespred use of computer-relted products hve given the ordinry humn being the potentil to perform numerous nd more complex tsks under shorter periods of time. However, in order to hrness the computer s power to ssist humns in their work, designers nd humn fctors specilists like need to understnd the technology while simultneously being sensitive to humn cpcities nd needs (Shneidermn, 1992). They need to pply humn fctors principles nd processes in mking humn-computer interctions suited, or more importntly, centred to the humn insted of the mchine - the computer. This is one of the resons how the Grphicl User Interfces (GUIs) cme bout. GUIs cn be defined s the use of direct mnipultion nd icons or other grphicl symbols on disply to interct with computer or computer-bsed devices (Strmler, 1993). Horton (1994) nd Böcker (1997) dded tht present-dy GUIs in computers utilise the so-clled WIMP interfce (windows, icons, mouse nd pointer). In generl, symbol is ny grphicl chrcter or other representtion which is intended to: () stnd for something else, (b) communicte use for n object/structure, or (c) communicte wht should or should not be done t given time or loction (Strmler, 1993). Grphicl symbols usully pertin to terms like icons nd pictogrms or pictoril symbols (Böcker, 1993). The former refers to 29

30 symbols tht re simple, concrete nd usully self-explntory of the ides, objects or functions they represent (Wood nd Wood, 1987). The ltter types re usully more bstrct, conveying messges by nlogy or symbolism. Thus, pictogrms require certin lerning processes to be understood. These terms would be discussed in more detil lter under the cognitive spects of symbol understnding. Nonetheless, the term icon is the most populr mong the terms nd hs become synonymous to ny smll visul symbol. Aside from being grphicl, the term icon lso refers to the non-linguistic representtion of n object or ction (Strmler, 1993). In ddition, when one tlks of computer systems nd documenttion nd grphicl interfces in generl, Horton (1994) described icons s the smll pictoril symbols used on the computer menus, windows nd screens. They represent the cpbilities of the computer system. They cn be ctivted nd bring forth these cpbilities into ction. Thus, compred to the commonly encountered trffic, directionl nd sfety signs nd symbols, icons nd pictogrms in computers nd other udio-visul devices re much smller nd usully ppering in sets. This pper dels with smll grphicl symbols. They would generlly be used here to pertin to both icons nd pictogrms. 1.3 Some spects of cognitive ergonomics of grphicl symbols Visul imge processing Knowing how people process visul imges is essentil in understnding how grphicl symbols cn be designed nd utilised better. Horton (1994) used two models to explin the visul imge processing of symbols. The first, he clled the nïve model of visul processing. According to this model, humn vision is likened to the cmer. Perception forms n interior imge tht is n exct copy of the scene in front of the eyes. This scene is then photogrphiclly recorded nd stored in memory. Although this model is simple nd intuitive, Horton clims tht this view is erroneous. If this were true, objects will only be recognised if they pper exctly similr s the first time they were seen. The other model ws clled the richer model of visul processing. He cited t lest five wys how it differed from the former model. Moreover, it more dequtely explined how symbols could be understood nd even remembered (Horton, 1994): - Perception filters visul inputs. Less thn 1/1000 of 1 percent of the informtion tken in by the retin mkes it through short-term memory - We remember not just one visul imge for the object but severl - In dditionl to visul memories, we remember the nme of the object nd its chrcteristics. All these memories re linked so tht recll of one triggers others - Much, perhps most, of wht we see comes not from the eyes but from the memory. We see mostly wht we lredy know nd hve seen before. Visul input seems 30

31 designed more to trigger the correct memories thn to tell us directly wht something looks like; nd - Wht we currently see nd wht we hve seen ffect how we interpret visul input nd wht we remember. We tend to seek out detils tht confirm our current interprettion of scene. Thus, the second model implies tht visul processing is much more complicted thn one would normlly presume. It involves the processes of visul perception, memory (short- nd long-term), tht re eqully ffected by personl experiences in the externl world The role of context in symbol mening After the concept of visul imge processing of symbols, we still need to know how symbols generte their mening. One cn esily gree tht grphicl symbols existing lone re of no mening. It is when grphicl symbol is used in prticulr context tht it cn trigger memories nd ssocitions in one s mind referred to s its mening (Horton, 1994, Edworthy nd Adms, 1996, Böcker, 1997, nd Dewr, 1999). Context is the sitution or scenrio wherein we encounter the symbol. It involves ll other things in the field of view tht cn dd or interfere with our perception of the symbol. Horton (1994) suggested tht context could be ny hint tht leds us towrd one prticulr interprettion of the symbol. Contexts cn thus be the djcent symbols, relted lbels or texts, other windows or displys, or other objects in the environment. We cnnot know how grphicl symbols will be interpreted without knowing the context wherein they will be seen (Wolff nd Woglter, 1998). Further, depending on the contexts nd the users, severl menings cn be generted by symbol: Icon i + context tj + viewer k = mening ijk Horton (1994) illustrted this by giving the exmple of the symbol shown in Figure 1. FIGURE 1. When shown lone, symbol s mening depends on the user s previous ssocitions. It could denote bttery for someone with n electricl engineering bckground. For someone involved with word processors, the sme symbol could be interpreted s centring lines of texts. However, when shown with other relted 31

32 symbols, the intended nd correct mening would become cler to most of the users. In this cse, its mening ws indeed the centring lines of texts (Figure 2). FIGURE The sign-signl-referent reltionship The model suggested by Fmilnt nd Detwiler (1993) is nother wy to see how symbols re relted to their intended menings or referents (Figure 3). They suggested tht informtion become encoded s "signl" tht cn be interpreted if it is mpped to "referent" to the rel world. The signl's chrcteristics depend on the medium of communiction. For exmple, the word "house" when spoken hs cousticl fetures, nd hs visul fetures when shown s picture. In turn, the "sign" is the reltionship between the signl nd its referent. This reltionship is seen to be successful when user cn unequivoclly refer to the signl's intended referent. Fmilnt nd Detwiler (1993) further dded tht signs could be "iconic" or "symbolic". They termed it iconic when the fetures of both the signl nd the referent intersect (ex. symbol of "trsh cn" to represent file ersure function - "throwing wy something"). It is symbolic when the sign reltionship is rbitrry nd hs to be lerned. Böcker (1997) further elborted tht visul symbols should not necessrily be pure icons nor pure pictogrms or symbols It is better to understnd the terms iconic nd pictogrphic or symbolic s endpoints or nchors on continuous scle. A symbol my then lie nywhere long such scle depending on its composition. Signl Sign reltion Referent Picture iconic or symbolic word object bstrct drwing bstrct concept FIGURE 3. Sign, signl, referent reltionship (Fmilnt nd Detwiler, 1993) Symbol clssifictions Together with the bove models of symbol perception, symbol chrcteristics hve lso become the bses of symbol clssifictions. In generl, symbols re ctegorised 32

33 whether they re imge-relted (representtionl), concept-relted (bstrct), or rbitrry (Modley, 1966, Dreyfuss, 1972, Tudor, 1994, nd Edworthy nd Adms, 1996). For exmple, in GUIs such s those in computer-bsed word processing progrms, the pictoril imge of printer is direct or concrete representtion of n object esily recognised. Abstrct or concept-relted symbols re those where n spect of the concept lends itself to symbolistion in wy tht it evokes the whole concept itself. The symbols for store nd retrieve s illustrted in Figures 4 nd 5 re good exmples of bstrct symbols. FIGURE 4. FIGURE 5. Arbitrry symbols re symbols whose menings re rbitrrily ssigned to the grphics. For exmple, when the symbol of n exclmtion mrk or question mrk (Figures 6 nd 7) is used, neither of them signifies ny spect of the rel world objects. Rther, mening hs been rbitrrily ssigned to ech one nd one hs to lern to understnd wht these menings re. FIGURE 6. FIGURE 7. When rbitrry symbols re used together with other symbols such s in GUIs, their grphic imges should be simple nd clerly distinct from the others whose concepts re closely relted or my be used under similr contexts (Edworthy nd Adms, 1996). Clssifying symbols bsed on bstrction or representtiveness is only one of severl methods of clssifying grphicl symbols (Böcker, 1997). For exmple, Dewr (1999) divided the symbols into five min ctegories: industril nd occuptionl (workplce); representing methods (mchines, instructions); mngement of public plces (trnsporttion, museums, hospitls); knowledge; nd prticulr ctivities (sports). Symbols my lso be clssified ccording to the following (Gittins, 1986, Horton, 1994, nd Böcker, 1997): 33

34 ) Surfce fetures: form (ssocitive vs. key), type (dynmic vs. sttic), colour (monochrome vs. coloured), nd loction (isolted vs. ttched) s differentiting fctors b) Metphors used: use of unifying metphor. This pertins to the use of knowledge cquired in different context to quickly lern nd use functions of new system being introduced. A good exmple is the desktop metphor in computer systems where the symbols from the office environment (files, folders, trshcn, telephones, etc.) re used to denote the different functions of the computer s operting system. c) Structurl Description: this ws suggested by My, Böcker, Brnrd nd Green (1990) bsed on their ppliction of method of structurl description developed in linguistics by Hllidy (1970). Bsed on the ltter, structurl description is forml wy of how to nlyse the components of n entity nd the different themes or themtic trnsitions tht cn be generted by different psychologicl subjects. An entity my be ny tsk or experience such s listening to someone tlk or looking t pictures. The different themtic trnsitions cn be used s indictors of the cognitive efforts required by ech subject to properly recognise the object in question. 1.4 The use of grphicl symbols throughout the times From pintings to printing press The use of signs nd symbols is not new. It is lso obvious tht the evolution of grphicl symbols is much tied to the history of humn communiction itself. As erly s 50,000 BC, symbols ppered s pintings or crvings on cves nd stone wlls, with depictions of humns dting s fr bck s 11,000 yers go (Dewr, 1999). Among the ncient civilistions, one could esily recognise the use of symbols in the writings of the Chinese, Egyptin nd Africn, nd even the erly Americn ntives. They hd used pictoril imges nd symbols to tell stories, myths nd even to provide instructions (Horton, 1994). In their book on wrning design, Edworthy nd Adms (1996) cited the work of Senner (1989) wherein the ltter discussed the history of written communiction s beginning with the use of pictoril symbols to represent simple ides. Soon, the simple stnd-lone symbols gve wy to bstrct writing systems like the cuneiform nd hieroglyphics. Erly in this process, he pointed out tht the pictoril representtions nd their use becme so stylised, eventully giving birth to the modern lphbets nd syllbries (syllbic scripts). This development implied tht those who hd lerned the symbols' menings nd their system of use would be the ones who could only use the written communiction system. Moreover, during these erly times, the previling technology ws hrnessed to fcilitte the use of such written communiction system. Simple technologies yielded simple devices for writing nd publiction. As technology dvnced, so did the use of the texts or modern lphbets in 34

35 communiction. The dvent of the printing press introduced the mss circultion of published documents. Books, newsppers, mgzines nd ll sorts of print mterils begn to be published ernestly nd numerously in countries touched by the technology of the printing press Modern technology, globlistion nd the return of the grphicl symbols The rise of modern technology certinly ws not confined in the printing press. Almost ll fcets of humn endevours hve been continully ffected by modern technology. For in wy, technology itself is n expression of the humn society s evolution. Thus, modern technology hs lso led to the growth in interntionl trvel nd communictions, s well s the increse in the use of computerised products by the common to (person). However, this globlistion of trvel nd products hve returned us to situtions wherein we need to convey simple concepts quickly nd efficiently, nd would not require extensive lerning (Edworthy nd Adms, 1996). We thus find ourselves going bck to the use of grphic symbols whenever it is deemed pproprite. Edworthy nd Adms (1996) trced the use of symbols in public informtion systems, industril sfety nd the present world of computer interfces to the gret success of using highwy signs. Then during the lst two decdes, grphicl symbols s prt of grphicl user interfces went into the upturn. A primry fctor behind this hs been the rpid development in microprocessor technology leding to the rise of computer nd informtion technology. The first microprocessor ws the Intel TM 4004, produced in 1971 (Encrt, 1997b). It ws originlly developed for clcultor contining 2300 trnsistors on 4-bit microprocessor. It could perform only 60,000 opertions per second. The first truly generl-purpose microprocessor ws developed in It ws the 8-bit Intel TM It contined 4500 trnsistors nd could execute 200,000 instructions per second. Modern microprocessors hve much greter cpcity nd speed. They include those mnufctured by Intel TM (Intel Pentium Series) nd by the Sun Microsystems TM (UltrSprc). Apple TM, IBM TM nd Motorol TM hve jointly developed the PowerPC microprocessors, while Digitl Equipment Corportion (Digitl TM ) hs the Alph microprocessors. All of these contin millions of trnsistors ech (Encrt, 1997b). It must lso be noted tht the technology of microprocessors nd integrted circuit fbriction is still chnging rpidly. Currently, the most sophisticted microprocessors contin bout ten million trnsistors. By the yer 2000, dvnced microprocessors re expected to contin more thn 50 million trnsistors, nd bout 800 million by 2010 (Encrt, 1997b). With the bove rpid dvncement in microprocessor technology nd its widernging pplicbility, lot of humn tools nd devices hve inevitbly become smller, multi-functionl, nd computer-bsed, with interfces tht re eqully smller with numerous control fetures. Accordingly, to properly hndle these hve led to the development of mixed interctive dilogue grphicl interfces for 35

36 personl computers nd worksttions such s Xerox s TM Str user interfce, the Apple Mcintosh TM operting system, nd the widely-used MS-Windows TM by Microsoft TM (Böcker, 1997). They generlly utilise toolbrs, pull-down menus, nd dilogue boxes contining either one or few buttons, to rrys of buttons with symbols s shortcuts to complex functions (Böcker, 1997). At present, the prolifertion of grphicl symbols is even no longer confined to the trditionl desktop nd notebook PCs. They re found mong the smller nd more compct PC vrints tht re clled plm-sized notebooks nd orgnisers. Even telecommunictions products re not spred, s exemplified by the incresingly populr mobile phone. In these products, buttons contin symbols representing the device s dditionl nd often more complex functions (i.e. text nd voice mils, telephone directories, nd personl settings). Severl resons hve been given why icons nd pictogrms hve come to be incresingly used gin. Most of these point to their dvntges over text-bsed messges themselves. Compred to text, symbols re more distinct (Mguire, 1985), fster nd esier to recognise (Collins nd Lerner, 1982), nd cn even reduce the likelihood of errors (Lodding, 1982). Horton (1994) pointed out those grphicl symbols such s icons used in computers nd computer-relted products mke these products go globl. In their pper on sfety pictogrms, Dvies, Hines, Norris nd Wilson (1998) mentioned tht in the light of the emerging Europen mrket, grphicl symbols hve become very ttrctive since they hve the potentil to cross the lnguge brrier. Messges my need not lwys be trnslted or explined in the 13 lnguges of the Union. Aside from the potentil of symbols to mke things usble under interntionl settings, some studies hve shown their usbility mong geritric people. As one gets older, contrst sensitivity decreses. However, Kline, Ghli, Kline nd Brown (1990) showed in their studies tht the visibility of symbols ws better thn their equivlent worded signs mong older subjects even under poor contrst conditions. 36

37 2 CURRENT PROBLEMS AND ISSUES 2.1 Restrictions on the dvntges of using grphicl symbols The use of grphicl symbols, however, is not devoid of problems. Foremost mong these is tht some symbols re simply not understood well (Brelsford, Woglter nd Scoggins, 1994). The dvntges cited bove were evident when the concepts being represented were well understood or concrete. Hence, the symbols used were representtionl or imge-relted types. The dvntges diminish when the ides become more bstrct. Snders (1998) further dded tht the superiority of symbols cn only be ttined if they contin smll mount of detil, re sufficiently distinct in shpe nd re unmbiguous. Regrding older people, Dvis et l (1998) cited the studies of Esterby nd Hkiel (1981) showing tht lthough symbols re esily recognisble, understnding their mening is nother mtter, nd tht it is generlly poorer compred to younger people. In ddition, most of the studies focused on symbols occurring individully nd/or bigger in sizes (i.e., informtionl nd directionl symbols in public plces such s highwys, irports, trin sttions nd hospitls). In turn, GUIs typiclly hve smller symbols (such s those in toolbrs nd smll disply screens of portble devices). Another problem with the use of grphicl symbols pertins to the reltionship between the rnge of symbols presented simultneously nd the contexts in which they re used. Edworthy nd Adms (1996) ptly pointed this out by stting tht lthough symbol is well understood in given context, this dvntge will only lst s long s the rnge of the symbols used in tht context does not led into higher chnces of confusion. For exmple, in dtbse or document filing progrms, folder with certin colour or mrk my indicte certin type of file. Such differentiting fetures should be distinct enough to help the user void confusions with other file types. 2.2 Stndrdistion problems The next set of problems regrding grphicl symbols pertins to stndrdistion. It is esy to understnd tht the prolifertion of grphicl symbols in n eqully incresing number of modern devices is problem in itself. Compounded by the growth in number nd diversity of trgeted users owing to globlistion, stndrdistion in design nd implementtion of symbols is obviously desirble. Dewr (1999) pointed out tht lthough efforts done by the Interntionl Orgnistion for Stndrdistion (ISO) hs led to some interntionlly ccepted symbols, there still remins high vriety of symbols used round the world. Symbol designs vry not only geogrphiclly but lso cross different pplictions (Olmsted, 1999). These re complicted by the fct tht n cceptble nd uniform level of comprehension when testing symbols is still lcking (Wolff nd Woglter, 1998, nd Dewr, 1999). Comprehension stndrds vry from 37

38 Interntionl Orgnistion for Stndrdistion s 67 percent (Zwg, 1989) to 85 percent of the Americn Ntionl Stndrds Institute (ANSI, 1993). Nonetheless, Dewr (1999) suggested tht it is more importnt tht cceptble levels of comprehension should consider the nture of the messge nd the consequences of filing to understnd it. 2.3 Symbol evlution issues The tendency to generlise tht symbols cn be esily designed nd understood by nyone is still high. If the person who cretes or designs symbol understnds its mening nd cn esily convince collegue tht it is very good representtion of certin referent or function, then the rest of the generl public will do so s well (Edworthy nd Adms, 1996 nd Rutter nd Beck, 1997). Reserches using nive popultions hve shown tht there is t lest 15% of the subjects who re not ble to correctly understnd the symbols being tested (Zwg, 1989). Although the 85% level of correct interprettions cn be considered s high, the extent of the implictions of hving 15% getting it wrong must lwys be weighed. With the current globlistion wherein technology nd its products trnscend ntionl borders, the extent nd levels of 'nive users' ssume greter implictions. Thus, the relevnce of empiricl testing grphicl symbols for their intended uses nd menings mong potentilly different trget user groups cnnot be overemphsised. Grphicl symbols hve to be tested for their intended uses nd contexts mong the intended users Current test methods With regrds to symbol testing, vrious methods hve been utilised to evlute icons nd pictogrms. Some of the methods frequently used were the following: ) ppropriteness tests, b) preference rtings tests, c) nming nd mtching tests d) comprehension tests, e) recognition nd recll tests, nd f) pired-comprison tests (Green nd Pew, 1978; Mckett-Stout nd Dewr, 1981; Jones, 1983; Rogers, 1986; Noln, 1989; Zwg, 1989; Mgyr, 1990; Slsoo, 1990; Stmmers, 1990; Clrke nd Brown, 1992; Lin nd Kreifeldt, 1992; nd Lin, Kreifeldt nd Chi, 1992). Webb, Sorensen nd Lyons (1989) described these nd other methods of evluting symbols s involving psychophysics, scling, recognition nd memory testing, s well s sttisticl modelling nd nlysis. The type of testing employed ws often dependent on the type of study being pursued. Generlly, ppropriteness testing ws the preliminry procedure to screen severl cndidte designs. Mtching studies determined how well the symbols performed s set nd their likelihood of being confused with ech other (Noln, 1989). Another type of test worth mentioning is the spontneous recll or identifiction test. It is lso oftentimes known s the open-ended type of test. The symbols being tested re shown to the subjects, individully or simultneously, nd they re sked to stte/write on spces provided wht they think ech symbol 38

39 men for them. This type of test is usully regrded s the yrdstick of ll other types of symbol recognition tests (Dewr, 1994 nd 1999, nd Wolff nd Woglter, 1998). This type of test is indeed very helpful nd informtive becuse it cn revel the ese of understnding of the symbol, s well s the extent of confusion nd types of errors tht the users mke. However, one big drwbck of this method is tht dt reduction cn be time consuming (Dewr, 1999). Intended use or context of the symbols should lso be cler to the subjects to void unnecessry or invlid confusions nd errors (Wolff nd Woglter, 1998) Use of severl test prmeters The use of the single test methods s described bove hs the dvntge of speed, simplicity nd ese in interpreting results. However, to properly evlute nd decide which mong severl lterntive symbols would perform best, severl tests re needed. Further, multiple test prmeters is deemed most suitble when the cndidte symbols re intended s prts of interfces of devices for interntionl use or for stndrdistion. For exmple, Mckett-Stout nd Dewr (1981) studied different versions of public informtion symbols using number of mesures nd deriving n "efficiency index" for ech symbol. In turn, the Interntionl Stndrds Orgnistion (ISO) cme out with the ISO It ws six-stge procedure for the development nd testing of public informtion symbols (Zwg, 1989). Its mjor portions were the comprehensibility judgement tests nd the comprehension tests. On the other hnd, the Interntionl Telecommunictions Union (ITU-T) hd F.910 (ITU, 1995). The Recommendtion F.910 endorsed symbol testing procedure composed of four prts. It involved the determintion of need for new symbols, the cretion nd evlution of the new designs, nd the selection nd pprovl The MIA - Multiple Index Approch An incresing number of icons nd pictogrms in most computerised personl, office nd even industril devices re smll nd pper in sets to represent series of functions. The Humn Fctors Technicl Committee of the Europen Telecommunictions Stndrds Institute (ETSI) evluted videotelephony symbols using multiple index pproch (MIA) which cn lso be used in testing symbols for other commercil products (Böcker, 1993; ETSI, 1993). This method ws lso developed in response to the issue tht most of the methods mentioned erlier were ment to test grphicl symbols tht were often lrge in size (trffic nd directionl symbols). These symbols re those usully found in public plces, nd ppering individully ( stnd lone ). The ETSI used MIA to test different sets of cndidte grphicl symbols (point-to-point videophone pictogrms) intended to pper together in smll displys nd were thus quite smll themselves (ETSI, 1993). Another rtionl behind the MIA ws the relevnce of tests reflecting ctul user scenrios. This view could be esily pprecited under the principle of contextulistion discussed erlier. To properly evlute symbol understnding, it 39

40 should be tested in contexts or situtions it is designed for use. For exmple, when symbols re designed to pper together, then they should be tested s such, in sets or groups. The intended device where they would be used (i.e. videophone) together with the referents to be tested should lso be properly presented nd explined. In ll, using MIA would enble the tester to collect seven indices tht cn help in the finl selection of symbols. The indices were divided into the objective nd subjective. The objective indices (lso known s performnce indices) were hit rtes, flse lrm rtes nd missing vlues. The subjective indices were subjective certinty nd suitbility, symbol nd symbol set preferences (Böcker, 1993). It ws emphsised tht lthough the test of ssocitiveness (hit rtes) ws the min index of performnce the other objective prmeters (flse lrm rte nd missing vlues) were lso essentil when testing symbol comprehension mong symbols ppering in sets. The subjective prmeters (subjective certinty, subjective suitbility nd preferences) would be used to decide between two or more grphicl symbols with similr performnce dt. This wy, the best set of symbols cn be decided using fctors other thn simple correct symbol-referent mtching. Thus, it is not difficult to pprecite the relevnce nd pplicbility of using such multi-fctoril pproch in evluting grphicl symbols cross diverse user groups cn be of gret help in evluting cndidte grphicl symbols for interntionl use The semntic differentil test The semntic differentil technique or SDT ws originlly developed to mesure the connottive mening of concept or ide (Osgood, Suci nd Tnnenbum, 1957; Edwrds nd Porter, 1972; Henerson, Morris nd Fitz-Gibbon, 1987). Its bsic purpose ws to nlyse concepts in question bsed on limited number of dimensions of mening. This involved subjective rtings of the stimuli on number of bipolr djectives belonging to three or four semntic differentil fctors. These were evlutive, potency, ctivity nd understndbility, with the first three known s the EPA fctors of Osgood (Osgood et l., 1957). Other semntic fctors cn ctully be derived s shown by Cron, Jmieson nd Dewr (1980). Concepts cn lso be compred with other concepts to evlute their similrity or difference in menings, or compre the sme sets of concepts mong different groups, including ntionl nd lnguge groups (Oppenheim, 1986). SDT hs lso been used to evlute pictogrms nd other grphic symbols (Dewr nd Ells, 1979; Cron, Jmieson, nd Dewr, 1980; nd Vor, Helnder, Swede nd Wilson, 1991). In order to knowing whether cndidte symbol is pproprite for use, it must be tested for comprehension. In their study of 20 trffic sign symbols, Dewr nd Ells (1979) considered the understndbility fctor s representing the extent to which the subject knows (fmilirity) nd comprehends the object or concept in question. Potency nd ction in turn were relted to comprehension nd subjective mening. Comprehension for ech symbol ws determined by computing the percentge of correct responses. 40

41 Semntic differentil scores for ech symbol were then clculted using men rtings of the djective pirs comprising the index of mening for ech fctor. Through rnk order correltion, they nlysed the reltionship between percentges of correct comprehension nd semntic differentil scores. They found out tht the evlutive nd understndbility semntic fctors might be useful in evluting the users perception nd understnding of the symbols. In contrst, Vor et l. (1991) used djective pirs commonly used by grphic designers (such s blnced - unblnced) but hd only one fctor ble to explin more thn 60 percent of the vrition. In series of studies, Lin nd his collegues employed the principles of semntic differentil testing towrds proposing tool tht computer symbol designers cn use in predicting informtion needed for evluting nd modifying them (Lin, 1992; Lin nd Kreifeldt, 1992; nd Lin et l. 1992). Insted of using the EPA fctors of Osgood, they derived three cognitive fctors, which they believe ffected symbol recognition nd comprehension. They tested its pplicbility in ctul symbol evlutions where the fctors they derived were ble to explin bout 87% of the vrinces. However, it would hve been eqully interesting hd they tested the method using prospective users lso. This wy, symbol evlution would hve hd more credence by using the users s evlutors. Still, SDT my be ble to give insights on the possible psychologicl fctors tht cn ffect symbol recognition nd comprehension. Ossner (1990) mentioned other studies showing tht the results of vrious methods my be ctully "relted to the semntic fetures inherent in the content/proposition without, however, determining them". SDT my thus be useful tool in grphic symbol evlution but more studies re needed using prospective users insted, which brings us to nother fctor ffecting symbol evlution - the subjects or respondents used in the tests Prospective users s subjects Another criticl fctor ffecting symbol evlution is the type of subjects involved in the studies. Tke the ordinry sitution wherein prospective users re choosing between different models of computer-relted product. Ech model hs the sme functions s the others but represented by different types of icons or pictogrms (one icon type or fmily, one model). Thus, s ech user proceeds in mtching the functions with the symbols, he/she gets to encounter how usble the product is for him. After trying the functions of product bsed on the symbols, the user gets to know if he did it right or not. This is usully the prt where users give comments tht the "keys" or the symbols or icons re difficult to recognise, or not comptible with the functions (referents). Thus, grphicl symbols s interfces in computer-bsed systems ply criticl role in their usbility (nd ppel) to the users. Users must be ble to see nd use symbols s pproprite, meningful, memorble nd lernble in order for them to fully utilise the products. However, icons nd pictogrms re generlly designed nd developed with western users in mind but trgeted for interntionl use. Moreover, the subjects used in some of the 41

42 trditionl methods discussed erlier could hrdly even be described s representing typicl users, such s industril design students, professionls, nd even communiction employees (Tudor, 1994) Culture nd symbol evlution Need for more studies on culture Finlly, users involvement become more crucil when users bckgrounds re vstly different s in estern vs. western users, where one culture develops the products nd their symbols, nd nother culture/s use/s them. Keesing (1974) defined culture s something tht lso influences the "publicly vilble symbolic forms through which people experience nd express mening". Although the influence of culture on technology hs been recognised, empiricl studies re still indequte (Abeyseker, Shhnvz, nd Chpmn, 1990, Sukviriy nd Morn, 1990, Shhnvz, 1991). Regrding symbol evlution itself, only recently hs cross-culturl studies strted to gin ttention (Lin, 1999, nd Olmsted, 1999). Studies focusing on third world countries, such s those in Asi, re still very few (Plocher, Grg, nd Chestnut, 1999). In his studies on culture, technology nd ergonomics, Shhnvz (1998) reveled tht pictoril signs nd symbols for public use re very much relted to culture. He suggested tht the best wy to consider culturl spects in the design is to dpt user-centred design pproch. The users needs nd bckgrounds hve to be reflected in the design nd evlution of the cndidte symbols. Böcker (1993) lso stressed the need to explin nd test the functions (represented by the symbols) in mnners reflecting ctul or typicl user scenrios. Tudor (1994), in using subjects who were potentil customers s well, pointed out the relevnce of involving ctul prospective users, including novices, of the products where the symbols re intended to be used. For exmple, if symbols re intended for users of diverse culturl bckgrounds, the choice of subjects must reflect this. Nothing exemplifies this better thn Southest Asi, or even Asi s whole A brief bckground on Southest Asi nd the ASEAN Asi is the lrgest of the world s seven continents. Together with the outlying islnds, it covers n re estimted to be 44,936,000-sq. km, or bout one-third of the world's totl lnd re. The Asin people ccount for three-fifths of the world's popultion. In the erly 1990s, Asi ws estimted to hve hd more thn 3.2 billion people living in the different countries (Microsoft Encrt, 1997). Bsed on culture, the Asin continent my lso be divided into two: tht which is Asin in culture (Est Asi, Southest Asi, nd South Asi) nd tht which is not (Asi of the former Soviet Union, nd Southwest Asi). Of prticulr interest is Southest Asi. It is region comprising the Indochinese nd Mly peninsuls nd severl nerby islnd groups. Southest Asi is bordered on the north by Chin; on the est by the South Pcific Ocen; on the south by the Indin Ocen; nd on the 42

43 west by the Indin Ocen, the By of Bengl, nd the Indin subcontinent. Southest Asi includes the countries of Brunei, Mynmr (formerly known s Burm), Cmbodi (Kâmpùché), Indonesi, Los, Mlysi, the Philippines, Singpore, Thilnd, nd Vietnm. The Assocition of Southest Asin Ntions (ASEAN) is regionl llince of seven independent countries of Southest Asi. ASEAN ws founded in Bngkok in August 1967 by Indonesi, Mlysi, Philippines, Singpore, nd Thilnd (Microsoft Encrt, 1997). Brunei joined fter ttining independence in Vietnm ws dmitted s the first Communist member in Its principl objectives were to ccelerte economic growth nd promote regionl pece nd stbility. A joint forum with Jpn ws estblished in 1977, nd coopertion greement with the Europen Community ws signed in In Jnury 1992, ASEAN members greed to estblish free-trde re nd to cut triffs on nongriculturl goods over 15-yer period beginning in Since then, nd except for the Asin economic crisis in , Southest Asi hs been experiencing rpid economic nd industril growth. With fst growing economy coupled with huge potentil mrket bsed on popultion much bigger thn Western Europe or Northern Americ, the influx of western technology nd its products re unvoidble The need to test nd use grphicl symbols in shrinking world The grphicl symbols only typify the current globlistion of technology nd its products. They hve grown in number nd types. They re being used cross n eqully incresing number of pplictions nd products. This is bly exemplified by the videophone s discussed in this study. The videophone s forerunner is the ordinry telephone tht fcilittes verbl communiction between two or more prties seprted by some distnce. Through technologicl developments, the videophone not only relys voice or udio inputs. It cn even send video nd dt between communicting prties. For dditionl functions under limited spce, symbols s interfce re thus very suitble. However, lthough grphicl symbols hve the potentil to ese the introduction nd use of complex devices such s the videophone, poorly designed symbols cn cuse the opposite effect. If they re trgeted for wide spectrum of user groups typified by the people of Asi (different ge, lnguge nd of course, culturl bckground), n empiricl evlutive method is essentil. It must consider the diversity of the prospective users (subjects) nd the multi-fctoril nture of symbol processing nd understnding s discussed bove. 43

44 3 RATIONALE AND OBJECTIVES OF THE PRESENT STUDY The bove discourse on the nture, impct, nd issues of grphicl symbols cn be summrised into few inter-relted fctors nd comprise the rtionle behind the current study. 1. The rpid pce in the development of the microprocessor hs led to the eqully rpid development of the computer nd informtion technology. 2. A concurrent globlistion of the economy hs likewise led to both the technology nd its products to trnscend ntionl borders. 3. The bove two fctors re reflected by the prolifertion worldwide of computerbsed industril nd personl tools tht re getting smller but with incresing functions nd complexity. 4. These hve lso led to the increse in use nd importnce of grphicl symbols in interfces (icons nd pictogrms) to mke the products understndble nd usble. Compred to the trditionl directionl nd trffic signs nd symbols, grphicl symbols in GUIs re smller nd pper in sets or groups. 5. Thus, symbol testing is very importnt to help determine which mong the severl lterntive symbols will be understood esily nd correctly by the intended users round the world. 6. When symbols re intended for interntionl use, using multiple performnce or test prmeters is essentil. In this thesis, performnce or test prmeters pertin to both objective nd subjective prmeters of testing (objective nd subjective dt). 7. At present however, cross-culturl studies deling with symbol evlution, especilly in Asi, re still megre in number nd fr in between. Thus, the min objective of this study ws to nlyse the understnding of smll grphicl symbols (pictogrms nd icons) mong different potentil user groups from estern (Asin) countries like Indonesi, Mlysi, Philippines, Thilnd nd Sri Lnk. For comprtive purposes, subjects from Finlnd, Sweden nd USA were utilised. Different performnce prmeters derived from severl tests were used to exmine how symbol understnding would differ or be similr cross different study groups. Bsed on the results, the study lso imed to formulte generl recommendtions regrding designing nd testing smll grphicl symbols cross different culturl groups. 44

45 4 HYPOTHESES The hypotheses tested were the following: () Subject groups (countries) would hve some significnt differences in interpreting the videophone symbols nd their intended referents or functions bsed on different test prmeters (hit rtes, confusions, missing vlues, preferences nd semntic scling). (b) Specificlly, Asin subjects would perform poorly compred to Europen subjects in identifying videophone symbols. (c) Videophone symbols tht re fmilir, simple nd concrete would perform better thn symbols tht re new, complex nd bstrct. 5 METHODOLOGY 5.1 Initil Study: Philippine-Swedish Study Respondents This study ws bsed on pilot study s discussed in Pper 1. One hundred subjects (50 ech from the Philippines nd Sweden) were involved in the initil study. The Filipino subjects hd men ge of 30.1 yers (Rnge = yers, S.D.=4.60). The Swedish subjects men ge ws 26.6 yers (Rnge = yers, S.D.=5.21). In both groups, the subjects were university students nd office employees of universities s well s smll- to medium-sized compnies ( employees/compny). All subjects hd t lest two yers experience using computers or computer-relted devices t work nd t home Mterils nd equipment In the form of booklets, severl grphicl symbols representing certin videophone referents or functions were tested. Twenty-three (23) referents nd 69 videophone symbols used by Tudor (1994) were selected (Figures 8 nd 9). All the referents nd videophone symbols were bsed on Pper 1. 45

46 REFERENTS ABSTRACT SYMBOLS CONCRETE SYMBOLS PROPOSED SYMBOLS 1. Achieve Dil Tone 2. Answer Ringing Cll 3. Cll Log 4. Conference 5. Dilpd 6. Drop 7. Help Specific 8. Help System 9. HFAI 10. Hold 11. Messge 12. Music On Hold FIGURE 8. The three sets of videophone grphicl symbols used in the initil study, continued next pge (from Tudor, 1994). 46

47 REFERENTS ABSTRACT SYMBOLS CONCRETE SYMBOLS PROPOSED SYMBOLS 13. Mute 14. Notes 15. Phone Cll Active 16. Retrieve 17. Ringer Select 18. Spekerphone 19. Speed Dil 20. Store 21. Switch Hook Control 22. Trnsfer Cll 23. Volume FIGURE 9. The three sets of videophone grphicl symbols used in the initil study, continued from previous pge (Tudor, 1994). The symbols were grouped into three types (sets): bstrct, concrete, nd proposed bsed on their generl ttributes. Sixty-two of the videophone symbols were designed in Englnd. The remining seven were designed in Jpn. These 47

48 were the proposed symbols for Volume, Achieve Dil Tone, Hold, Music on Hold, Mute, Store, nd Spekerphone. The questionnire hd two prts: recognition/subjective certinty tests, nd semntic differentil rtings using six bipolr scles (Simple-Complex, Concrete-Abstrct, Meningful-Meningless, Fmilir-Strnge, Relted-Unrelted, nd Shrp-Dull). These bipolr djectives were selected bsed on studies by Dewr nd Ells (1979), Cron, Jmieson, nd Dewr (1980), nd Vor, Helnder, Swede nd Wilson (1991). Questionnires were produced per symbol set with the symbols nd their referents presented in rndom orders Procedure Three orderings of the videophone symbol sets were generted nd rndomly ssigned to the subjects during the recognition/subjective certinty tsk. The order of the referents ws likewise rndomised when presented. In the recognition tests, the videophone symbols nd referents were presented nd to be mtched correctly. For ech recognition test, confidence judgements on the nswers were indicted using 7-point scle. After one week, the semntic differentil testing ws performed. The symbols were gin presented. This time, they were to rte ech symbol using the six bipolr djectives. The sme tsks were done for the three symbol fmilies. Thus, ech subject performed 69 recognition/subjective certinty tests, nd 69 semntic differentil tests (6 bipolr scles ech), for the three symbol sets. Likewise, in the semntic differentil tests, the symbol sets nd referents were lso presented in rndom order mong the subjects. The response vribles in SDT were in the form of rtings using 7-point scles nchored in six bipolr djectives. The subjects were sked to rte ech symbol using the six bipolr scles. Spermn's rnk order correltion ws used to study the reltionship between hit rtes nd semntic differentil scores of the different videophone symbols. 5.2 Min Study: Southest Asi, Europe nd USA (Ppers 2 to 7) In the design of the methodology for the min study, severl fctors were considered. It took into ccount the recommendtions by Tudor (1994) in using prospective users s subjects nd considering typicl user scenrios for the tests. It lso dopted the MIA steps used by Böcker (1993). Specificlly, since culturl diversity ws prime fctor (Shhnvz, 1998) becuse the videophone symbols were lso intended for interntionl use, subjects from different countries were trgeted. This lso bolstered the considertion of using multiple prmeters of testing symbol understnding cross such diverse user (subject) groups. 48

49 5.2.1 Respondents Three hundred fifty-seven subjects from Southest Asi, Finlnd, Sweden nd USA were involved in the min study (187 mles nd 170 femles). The overll ge rnge ws yers old, with men of 25 (S.D. = 6.8). Tble 1 shows the gender nd ge distribution per country. Two hundred forty of them were from five Asin countries (forty-eight subjects per country). These countries were Indonesi, Mlysi, Philippines, Thilnd nd Sri Lnk. They were students, employees nd professionls (doctors, lwyers nd engineers), ll of whom were computer-literte or hd t lest experienced using computers or computer-relted products for the pst 2-5 yers t the time of the study. At the time of the study, none of the subjects hd experienced using videophone. Furthermore, forty-eight of the subjects were from Sweden, forty from USA nd 29 from Finlnd. The ltter (Finnish subjects) were specil cse involving elderly subjects (Pper 6). The former two pertined to subjects who were students nd employees in Sweden nd the US respectively (Pper 5), nd were similr in demogrphic trits to the Asin subjects (except of course, for ntionlity or country). TABLE 1. Gender nd Age distribution, in yers, per country, n = 48 per country, except U.S.A. where n = 40. Subject's Country Sex N Minimum Mximum Men Std. Dev. Indonesi Mle Femle Mlysi Mle Femle Philippines Mle Femle Thilnd Mle Femle Sri Lnk Mle Femle Sweden Mle Femle USA Mle Femle

50 5.2.2 Mterils nd equipment In Ppers 2 to 7, three sets of videophone symbols gthered from studies by the Europen Telecommunictions Stndrds Institute (ETSI) were tested in the form of questionnires (Figure 10). Set 1 ws the trget set nd contined the symbols dopted by ETSI s stndrd. The two other sets (Sets 2 nd 3) cted s distrcter sets. Ech symbol set contined seven symbols representing seven videophone functions (referents). The seven videophone functions tested were the following (Böcker, 1993; ETSI, 1993): ) Videophone/Telephone: upgrding / downgrding the cll b) Cmer on/off: turning on nd off picture trnsmission c) Microphone on/off: turning on nd off sound trnsmission d) Selfview on/off: turning on nd off selfview function e) Document Cmer on/off: switching between document nd person cmer f) Still Picture on/off: turning on nd off screen freeze g) Hndsfree on/off: switching between hndset nd hndsfree modes Procedure Testing of the subjects were done in universities, hotels nd offices locted in the cities of the countries involved in the studies. These were: 1. Metro-Mnil, Philippines 2. Bngkok, Thilnd 3. Bndung, Indonesi 4. Srwk, Mlysi 5. Perdeniy, Sri Lnk 6. Luleå, Sweden 7. Oulu, Finlnd 8. Sn Frncisco, Cliforni, nd Florid, USA Both questionnires nd orl instructions were mde vilble in English for subjects from Mlysi, Philippines, Sweden nd USA. For the other countries, they were lso mde vilble in the country s mother tongue (ntive lnguges). 50

51 FIGURE 10. Grphicl symbols used in the min studies (Ppers 2 to 7) s bsed on Böcker (1993) for the Europen Telecommunictions Stndrds Institute (ETSI, 1993). Tests were performed in smll groups in ech country, lsting for bout minutes. Subjects were rndomly given one of three versions of the questionnires. The subjects were shown n illustrtion of videophone. Its generl functions were then discussed. Afterwrds, instructions were given on how to go bout the different test prts. Questions were entertined prior to the dministrtion of the tests. Emphsis ws given on voiding omissions in trils in order to get bck to them lter. The order of the tests would lso be strictly followed; tht is, Prt One followed by Tests Two, Three, nd Four. In the spontneous identifiction test (Prt One), ech pge contined one set of seven videophone symbols rendering three pges for the three symbol sets. On spces provided for, they would write wht videophone function they thought ws represented by ech symbol. They were lso sked to rte the level of certinty for ech of their nswers using seven-point rting scle (from very certin to very uncertin ). In the cued response test (Prt Two), the subjects first red referent (function) nd its description. Then they hd to choose one symbol from set of seven symbols which they thought best represented the referent in question. Ech pge contined one referent-description nd one set of symbols. They were lso sked to rte their certinties for their nswers using the 7-point rting scles. There were seven videophone referents tested on three sets of symbols rendering totl of twenty-one trils in Prt Two. In the third test, five semntic bipolr scles similr to the first study were used to evlute ech symbol together with its correct referent, for totl of twenty-one SDTs. In Prt Four, the three symbols (one for ech set) together with the referent 51

52 they represented were shown. The subjects hd to choose one symbol they thought best represent tht referent. Next, with the three symbols sets presented together, they hd to choose the set they prefer most. In ll, the subjects would choose seven symbols to represent the seven referents nd one symbol set they preferred most Experimentl design The whole min study ws 2 x 7 x 3 (Occuption x Country x Symbol Set) repeted mesures design with 2-between nd 1-within fctors set-up. Between fctors were those of occuption (students nd employees/professionls) nd country (Indonesi, Mlysi, Philippines, Thilnd nd Sri Lnk). The within fctor consisted minly of the symbol sets (Sets 1, 2 nd 3). The videophone symbols were bsed on the studies by Böcker (1993; ETSI 1993), with Set 1 s the proposed set of symbols for seven bsic functions to be found in videophone. In ll four tests, the order of the symbol sets ws counterblnced cross ll subject groups. Three orderings of the symbol presenttion nd referents were generted nd rndomly distributed in ech subject group. The response vribles from the four tests were the following: () Hits nd Misses (men percentges of correct nd incorrect identifiction nd recognition in Prts One nd Two, using ISO 9186 of 67% correct recognition s generl guide of cceptble level) (b) Confusions (flse lrms) for ech videophone symbol (totl numbers nd frequency distributions) (c) Missing vlues (frequencies of No Answers in Tests One nd Two) (d) Subjective certinty rtings using 7-point scles for nswers in Tests One nd Two (frequency distributions nd men rtings) (e) Semntic differentil rtings of ll symbols in the three sets using five 7-point scles ech nchored in bipolr djectives (frequency distributions nd men rtings, Prt Three) (f) Symbol nd symbol set preferences (percent distribution, Prt Four) 52

53 6 RESULTS AND DISCUSSION 6.1 Initil Study: Philippine-Swedish Study Tble 2 summrises the results for the recognition-mtching tsks. Concerning the referents nd both countries in generl, only 4 of the 23 referents hd symbols with t lest 67% men recognition in ll three sets ( Conference, Dilpd, Music on Hold, nd Volume ). All the other icons hd rtes less thn 67% regrdless of which icon set they belonged. Agin, since the hit rtes (proportion of nswers considered s correct ) were bsiclly binomil in distribution, rcsine trnsformtions were performed in order to meet ssumptions of normlity nd tht multifctor ANOVAs would be vlid Min effects nd interctions Between sets, the Proposed Icons hd the most number of high (t lest 67%) recognition rtes (Phil.: 11, Sweden: 8), followed by the Concrete Icons (Phil: 9, Sweden: 8), nd Abstrct Icons lst (Phil.: 6, Sweden: 4). Between these icons, the results of multifctor ANOVA with Set s min effect showed tht for the referents Answer Ringing Cll, Messge, Mute, Retrieve, Ringer Select, Speed Dil, nd Trnsfer Cll, the Concrete nd/or Proposed icons hd significntly higher recognition rtes compred to the Abstrct versions (F(2, 6) nd p<0.05). When the Country fctor ws tken s the other min effect, the Filipino subjects scored significntly higher recognition rtes thn the Swedish subjects for the icons representing Answer Ringing Cll, Retrieve Cll, Speed Dil nd Trnsfer Cll. However, except for the ltter cse, these rtes were generlly below the 67% correct recognition of ISO For interction effects between country nd icon set, in both subject groups recognition rtes were generlly significntly higher with either the Concrete or Proposed icons (F(2, 6), p<0.05) for their symbols for Retrieve Cll, Speed Dil nd Trnsfer Cll Certinty rtings The countries were likewise compred regrding certinty rtings to their nswers in the recognition tsks using Mnn-Whitney tests (Appendix 1). For the bstrct set, the following icons showed significnt differences in certinty rting between the two countries: Answer Ringing Cll, Conference Cll, Dilpd, Drop, Help System, nd Speed Dil. For the Concrete Set, these were icons for Dilpd, Help Specific, HFAI, Hold, Music on Hold, Mute, Notes, Ringer Select, Store, Switch Hook Control, Trnsfer, nd Volume. For the Proposed Set, significnt differences in certinty rting scles were noted for Dilpd, Help Specific, Help System, nd Phone Cll Active. 53

54 TABLE 2. Study 1: Recognition rtes in percent (first rows, bold-fced: bove 67%) nd certinty rtings (second rows) for the 23 Referents by subjects from Philippines nd Sweden (N=100). Note: Phil. = Philippines, Swed. = Sweden. Abstrct Icons Concrete Icons Proposed Icons Referents PHIL. SWED. PHIL. SWED. PHIL. SWED. 1. Achieve Dil Tone Answer Ringing Cll Cll Log Conference Dilpd Drop Help Specific Help System HFAI Hold Messge Music On Hold Mute Notes Phone Cll Active Retrieve Cll Ringer Select Speed Dil Spekerphone Store Switch Hook Control Trnsfer Cll Volume

55 Among the four referents with t lest 67% correct recognition in ll sets, nd except for the concrete version of Music on Hold, the Swedish subjects exhibited significntly higher rtings thn their Filipino counterprts. But when compring icons for referents with less thn 67% recognition nd where the two countries significntly differ in their recognition rtings, the Filipinos hd the higher certinty rtings for Answer Ringing Cll, Ringer Select, Speed Dil, nd Trnsfer Cll. Swedish subjects hd significntly higher rtings only for the concrete icon for Mute. There were no differences between their rtings of icons Messge nd Retrieve Semntic Differentil Scles Tbles 3 to 6 contined the different levels of correltion between the icons of the three icon sets nd the semntic scles for ech country. TABLE 3. Correltions (rnk order) between semntic differentil scores nd percentge correct identifiction (N=100); *p < Abstrct Icons Concrete Icons Proposed Icons Referents PHIL. SWED. PHIL. SWED. PHIL. SWED. 1. Achieve Dil Tone Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 2. Answer Ringing Cll Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 3. Cll Log Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 4. Conference Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity.419* * * *.409* *.366* * *.379*.365* *.365* * *.363*.513*.574* *.535*.544*.465*.548* *.441*.443*.490*.580*.581* * *.504*

56 TABLE 4. Correltions (rnk order) between semntic differentil scores nd percentge correct identifiction (N=100); *p < 0.01, continued. 5. Dilpd 6. Drop Abstrct Icons Concrete Icons Proposed Icons Referents PHIL. SWED. PHIL. SWED. PHIL. SWED. Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 7. Help Specific Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 8. Help System Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 9. HFAI 10. Hold 11. Messge Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity *.702*.602*.659*.315*.598*.472*.478*.367* *.404*.479* 434*.502* *.433* *.440* * * * *.407* *.695*.418*.425*.469*.253*.752* *.411*.610* *.451*.622*.554*.429*.539* *.390* * * * * * * *.377*.404* *.80* *.388*.385* *.604*.619*.732*.746*.698*.735* *.552*.547*.368*.459* * * *.429*.418*.483* *.366*.393* * *

57 TABLE 5. Correltions (rnk order) between semntic differentil scores nd percentge correct identifiction (N=100); *p < 0.01, continued. Abstrct Icons Concrete Icons Proposed Icons Referents PHIL. SWED. PHIL. SWED. PHIL. SWED. 12. Music On Hold Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 13. Mute 14. Notes Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 15. Phone Cll Active Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 16. Retrieve Cll Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 17. Ringer Select Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity * *.392*.370* *.388*.493*.557*.467* *.449*.563* *.406*.617*.523*.734* * * * *.400*.538*.375* * * *.590*.462*.523*.635*.428* * * * *.397* * * * * * * * * *.408*.443*.377*.451* * * *.421* * * * * *

58 TABLE 6. Correltions (rnk order) between semntic differentil scores nd percentge correct identifiction (N=100); *p < 0.01, continued. Abstrct Icons Concrete Icons Proposed Icons Referents PHIL. SWED. PHIL. SWED. PHIL. SWED. 18. Speed Dil Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 19. Spekerphone Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 20. Store Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 21. Switch Hook Control Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 22. Trnsfer Cll Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity 23. Volume Concreteness Fmilirity Meningfulness Reltedness Shrpness Simplicity.448* *.614*.574*.612*.627*.613*.788*.805*.652*.786* *.456*.515*.496* * *.376*.395*.387*.420* *.433* *.714*.525*.380*.580*.452* * * * * *.398*.565* *.378*.421* *.496*.701*.634*.428*.459* *.627*.518*.528*.485*.584* * * * *.384*.391*.397* When the countries were compred in their semntic scles of the icons, the Filipinos gve consistently higher rtings on lmost ll scles compred to the Swedish subjects (Appendix 2). Further, the semntic scles for concreteness,

59 simplicity nd fmilirity hd the most number of significnt correltions to the rte of correct recognition of the icons. The results of the initil study reveled few but very importnt findings. First, the subjects poorly recognized mjority of the icons thn expected. This my men tht icons s interfce per se without ny other visul cue, id or even prior prctice to their use is not lwys helpful, s shown in n erlier study (Pper 1). They need to be tested thoroughly to be sure tht they re esily nd correctly recognized. Second, lthough very few differences were noted between countries regrding their recognition rtes of the icons, significnt differences were present in their certinty nd semntic rtings. Filipinos tend to give higher rtings compred to the Swedes who generlly were more conservtive nd gve lower rtings. Filipinos, like most Asins tend to give higher rtings thn westerners. This could represent kind of culturl bis on evlution tsks nd need to be studied further. Nonetheless, it turned out tht concreteness nd simplicity of the icons were eqully, if not more importnt. These results supported other studies pointing to the relevnce concreteness of the design nd the function in their recognition. It is esier for subjects to recognize icons or symbols tht represent concrete icons compred to bstrct ones. Lstly, regrding culture nd compred to studies done by Tudor (1994) using the sme icon sets mong Americ subjects, the Filipino subjects (followed by Swedish subjects) hd the highest recognition scores for most of the icons designed in Jpn. This could be reflection of the Philippine socio-economic condition where mjority of current computerised nd electronic pplinces is imported from Jpn. Icons or other grphicl symbols re commonly found in these products. Americns on the other hnd re obviously more kin to be exposed to products they hve mde. Although USA still rnks s the mjor trding prtner of the Philippines followed by Jpn, the ltter rnks first in terms of the computer nd telecom products mrketed nd used loclly. Thus, this renders the view tht culturl fmilirity nd mening of icons cn ffect their usbility when pplied s interfce tools in new or modern technologies. Admittedly, there re limittions in this pilot study. First, distinct group of potentil users (educted nd computer literte) ws used s subjects. The tests used in ll these studies were cued-response tests. The trget functions were presented nd explined nd subjects were sked to mtch the pproprite icons. Dewr (1994) hs pointed out tht mtching tests or multiple tests could gretly influence results of symbol recognition tests. He considered the spontneous identifiction or open-ended tests s the min stndrd of mesuring symbol understnding nd thus must be the bsis of compring other symbol test results. Still, most of the icons in the two studies hd very low recognition scores (<67%). To further study the observtions mde regrding the possible role of culture s well s other possible group differences in recognising symbols, similr nd other types of tests would be needed using different culturl 59

60 groups. Consequently, subjects from Asi, Europe nd Americ were then trgeted for the next series of experiments. 6.2 Min Study: Southest Asi, Sweden, USA, Finlnd (Ppers 2 to 7) The next series of tests involved 240 subjects from 5 Asin countries, 48 subjects from Sweden, 40 subjects from the United Sttes, nd 29 subjects from Finlnd (Totl number of subjects: 357). Compred to the erlier study, the min study used multiple test prmeters to evlute three sets of grphicl symbols representing the sme device (videophone) but for different referents. The different prmeters were hit rtes for both spontneous identifiction nd cued response tests, subjective certinties, flse lrms or confusions, missing vlues, preferences, nd semntic differentil rtings Exposure to computer-relted devices The min study ws first done in the five Southest Asin countries (Ppers 2 to 4) followed by tests in Sweden, Finlnd nd the USA (Ppers 5 to 7). Dt on subjects exposure to computer-relted devices s well s opinions regrding technology were lso obtined. Tbles 7 to 9 contin dt on the use of computer nd/or computer-relted devices (i.e. modern udio-video nd telecommuniction devices) mong the subjects. Except for subjects from Sri Lnk, lmost ll respondents (>90%) hve used computer-relted devices for the pst 2-5 yers. TABLE 7. Percent of subjects using computer or computerrelted devices (n = 48 per country, except U.S.A. where n = 40). Country Count Percent Indonesi Yes Mlysi Yes Philippines Yes No Totl Thilnd Yes Sri Lnk Yes No Totl Sweden Yes USA Yes Concerning ctul use of computer, t the time of the study t lest 68.8% of ll subjects per country use computers from occsionlly (t lest once week) to often (t lest once dy) (Figure 11). Indonesin subjects followed by Sri Lnkn subjects use the computer lest often, bout 68.8% nd 72.9% respectively. All other country groups hd more thn 90% of the subjects using computers t lest once week. 60

61 TABLE 8. Percentge distribution of computer use (n = 48 per country, except U.S.A. where n = 40). Rrely Occsionlly Often Not Applicble Totl Indonesi Count % Mlysi Count % Philippines Count % Thilnd Count % Sri Lnk Count % Sweden Count % USA Count % Rrely Occsionlly Often Not Applicble Frequency of computer use IND. MAL. PHI. THA. S.L. SWE USA Country FIGURE 11. Percentge distribution of computer use per country in percent (n = 48 per country, except U.S.A. where n = 40). 61

62 Regrding the type of computer interfce used by the subjects, ll country groups use both keybord nd windows-bsed interfces (Tble 9) with the ltter (windows) s the predominnt one being used. TABLE 9. Computer interfce used (n = 48 per country, except U.S.A. where n = 40). Country Keybord Grphics/ Not -bsed Windows-bsed Both Applicble Totl Indonesi Count % Mlysi Count % Philippines Count % Thilnd Count % Sri Lnk Count % Sweden Count % USA Count % Attitudes towrds technology Three bsic rting scles were used to compre the subject groups ttitudes towrds technology (ETSI, 1993). Tble 10 nd Figures contin the results of the rtings. Mjority (>50%) of the subjects in ech country think tht technology gives more dvntges nd mkes life esier in generl. However, most Swedish subjects greed tht some spects of technologicl progress re worrying compred to most Asin nd U.S. subjects who thought otherwise. 62

63 TABLE 10. Men rtings of ttitudes towrds technology (n = 48 per country, except U.S.A. where n = 40). Country Men S.D. Indonesi More Advntges Esier Life Technology Worrying Mlysi More Advntges Esier Life Technology Worrying Philippines More Advntges Esier Life Technology Worrying Thilnd More Advntges Esier Life Technology Worrying Sri Lnk More Advntges Esier Life Technology Worrying Sweden More Advntges Esier Life Technology Worrying USA More Advntges Esier Life Technology Worrying More Advntges Percent IND. MAL. PHI. THA. S.L. SWE. USA Countries FIGURE 12. Percentge distribution of ech country s rtings on technology s hving more dvntges thn disdvntges (1 = do not gree t ll, to 5 = totlly gree). 63

64 Esier Life Percent IND. MAL. PHI. THA. S.L. SWE. USA Countries FIGURE 13. Percentge distribution of ech country s rtings on technology mking life esier (1 = do not gree t ll, to 5 = totlly gree). Technology Worrying Percent IND. MAL. PHI. THA. S.L. SWE. USA Countries FIGURE 14. Percentge distribution of ech country s rtings tht current technology is worrying (1 = do not gree t ll, to 5 = totlly gree) Hit rtes Descriptives Hit rtes in Prt One referred to instnces when subjects wrote the word or words tht correctly stte or described the function or referent of ech prticulr symbol. In turn, Prt Two's hit rtes were instnces when subjects ssocited the referent in question to the correct videophone symbol mong set of other videophone 64

65 symbols presented. Tbles 11 nd 12 contin the results of the hit rtes for Prt One (Spontneous identifiction tests) nd Prt Two (Cued-response tests) respectively. Bsed on the men hit rtes lone in both Prts One nd Two, the grphicl symbols of Set 1 performed best individully nd s set. However, Prt One's hit rtes were generlly lower compred to Prt Two, minly becuse the former ws the hrder type of test. When the 67% of ISO 9186 ws used s nother bsis of performnce, very few of the symbols, especilly in Prt One met this criterion. Between individul symbols, very few reched the minimum 67% criterion. Only Set 1's symbols for "cmer" (68.2%) nd "microphone" (77.4%), nd Set 3's "microphone" (77.4%) hd overll men hit rtes bove 67% in Prt One. However, between sets, none of the 3 sets reched 67% (overll men for ech set in Prt One ws 40.5%, 16.2%, nd 36.5% respectively, Tble 11). TABLE 11. Prt One hit rtes (%) per country (n = 48 per country, except USA, where n = 40). SET 1 Ind. Ml. Phi. Thi. S.L. Swe. U.S.A. Mens [1] Cmer [4] Doc. Cmer [7] Hndsfree [10] Microphone [13] Selfview [16] Still Picture [19] Videophone Mens SET 2 [2] Cmer [5] Doc. Cmer [8] Hndsfree [11] Microphone [14] Selfview [17] Still Picture [20] Videophone Mens SET 3 [3] Cmer [6] Doc. Cmer [9] Hndsfree [12] Microphone [15] Selfview [18] Still Picture [21] Videophone Mens Legend: Ind.=Indonesi, Ml.=Mlysi, Phi.=Philippines, Thi.=Thilnd, S.L.=Sri Lnk, Swe.=Sweden, U.S.A.=United Sttes of Americ. 65

66 TABLE 12. Prt Two hit rtes (%) per country (n = 48 per country, except USA, where n = 40). SET 1 Ind. Ml. Phi. Thi. S.L. Swe. U.S.A. Mens [1] Cmer [4] Doc. Cmer [7] Hndsfree [10] Microphone [13] Selfview [16] Still Picture [19] Videophone Mens SET 2 [2] Cmer [5] Doc. Cmer [8] Hndsfree [11] Microphone [14] Selfview [17] Still Picture [20] Videophone Mens SET 3 [3] Cmer [6] Doc. Cmer [9] Hndsfree [12] Microphone [15] Selfview [18] Still Picture [21] Videophone Mens Legend: Ind.=Indonesi, Ml.=Mlysi, Phi.=Philippines, Thi.=Thilnd, S.L.=Sri Lnk, Swe.=Sweden, U.S.A.=United Sttes of Americ. In Prt Two, there were more symbols with hit rtes t lest 67%. Set 1, side from being the only set with n overll men hit rte bove 67% (71,8%), hd the most number of symbols meeting this criterion (5 of 7 symbols). These were Set 1's symbols for cmer (72.4%), document cmer (87.6%), microphone (83.8%), selfview (68.6%), nd videophone (91.3%). Set 3 followed with 4 of its 7 symbols hving hit rtes t lest 67%. These were document cmer (67.4%), microphone (84.3%), selfview (69.5%), nd videophone (82.4%). Set 2 performed the worst both s set nd with its individul symbols. Only its symbol for document cmer (74.5%) hd n overll men hit rte bove 67%. When studying the hit rtes between country groups, ll 7 countries fred very poorly in ll 3 sets during Prt One, with ll of them hving men hit rtes below 50% (Tble 11). In Prt Two, the results were comprtively better. Only one country group (Sri Lnk) hd men hit rte below 67% in Set 1. However, for 66

67 Set 3, only Sweden nd USA hd men hit rtes tht were bove the 67% criterion. Agin, none of the country groups performed well when using Set 2 (ll below the 67% criterion) Sttisticl Anlyses The ptterns observed in the descriptives were generlly confirmed by sttisticl nlyses. For exmple, regrding tests (Prt One versus Prt Two) s within fctor, hit rtes in the cued response tests (Prt Two) were significntly higher (t F(1, 321), p<0.05) thn in the spontneous identifiction tests (Prt One). Between symbol sets, Set 1 hd significntly higher hits in both tests for ll referents except for "hndsfree" nd "selfview" (F(2, 642), p<0.05). On the other hnd, Set 2 grnered the lowest hits in both Prts One nd Two of the test. Between countries, there were significnt differences (F(4, 321), p<0.05) in hit rtes for symbols for cmer, document cmer, hndsfree, selfview, nd videophone. In these symbols, Sri Lnkn subjects generlly hd the lowest hits compred to the other six countries. Focusing on the Swedish nd Americn subjects, some similrities nd differences were likewise noted. For exmple, in the identifiction tests (Prts One nd Two), Set 1 symbols generlly hd the higher hit rtes in both countries. However, both countries performed poorly in the spontneous identifiction tests (Prt One). For exmple, subjects from both countries mnged to ttin the Orgniztion for Interntionl Stndrdiztion s minimum correct recognition rte of 67% in only five of the 21 symbols tested. Compred to the Southest Asin studies using the sme symbol sets, lthough the Swedish nd Americn subjects fred quite better thn their Asin counterprts especilly in Prt One, the scores were still mostly below the criticl 67%. As discussed erlier, spontneous identifiction tests simulte, lbeit, in limited mnner, the first-time encounter of n interfce. The poor results thus strongly suggest tht without sufficient lerning opportunities nd lerning ids, symbols my be hrder to understnd by even subjects or trgeted users from western countries Confusions nd missing vlues Confusions or flse lrms were instnces when symbol ws ssocited to the wrong referents. They were derived during the cued response test in Prt Two (Tbles 13 to 19). For exmple, in the Indonesin group during the cued-response test nd testing Set 1 symbols, when the referent cmer ws presented, 10% of the subjects wrongly selected symbol for document cmer. Tbles 13 to 19 clerly showed tht Set 2 hd the most instnces of flse lrms nd Set 1 the lest. Across symbol sets, selfview nd videophone symbols hd the highest number of flse lrms. 67

68 TABLE 13. Confusion Mtrices nd Missing Vlues, Indonesi, n = 48. Referent Presented Symbol Set 1 Set 2 Set 3 Selected Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Missing Vlues TABLE 14. Confusion Mtrices nd Missing Vlues, Mlysi, n = 48. Referent Presented Symbol Set 1 Set 2 Set 3 Selected Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Missing Vlues TABLE 15. Confusion Mtrices nd Missing Vlues, Philippines, n = 48. Referent Presented Symbol Set 1 Set 2 Set 3 Selected Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Missing Vlues

69 TABLE 16. Confusion Mtrices nd Missing Vlues, Thilnd, n = 48. Referent Presented Symbol Set 1 Set 2 Set 3 Selected Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Missing Vlues TABLE 17. Confusion Mtrices nd Missing Vlues, Sri Lnk, n = 48. Referent Presented Symbol Set 1 Set 2 Set 3 Selected Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Missing Vlues TABLE 18. Confusion Mtrices nd Missing Vlues, Sweden, n = 48. Referent Presented Symbol Set 1 Set 2 Set 3 Selected Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Missing Vlues

70 TABLE 19. Confusion Mtrices nd Missing Vlues, U.S.A., n = 40. Referent Presented Symbol Set 1 Set 2 Set 3 Presented Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Missing Vlues TABLE 20. Missing Vlues or no nswers in percent (%) in Cued Response Tests, per symbol set (n = 48 per country). CAMERA DOC. CAMERA HANDS- FREE MICRO- PHONE SELF- VIEW STILL PICTURE VIDEO- PHONE Countries Indonesi Mlysi 2 Philippines Thilnd Sri Lnk Sweden U.S.A When symbols pper simultneously or s set nd the min tsk is to pick out which mong these symbols correspond to desired function or object, there re other possible outcomes side from hit. These re the so-clled non-hit outcomes or prmeters such s misses, confusions nd missing vlues (filure to respond or no nswer). The wrong nswers during the cued response tests were nlysed using confusion mtrices (Tbles 13 to 19). In this prticulr prt of the study, tbultions were mde on the number of instnces of confusions the instnce wherein ech symbol ws selected in the context of different referent. The results showed tht in the seven countries studied, the symbols of Sets 1 nd 3 hd the highest hit rtes (with the former usully with higher hit rtes thn the ltter) with hits bove ISO s 67%. The results of the confusion mtrices could be used to study which mong the symbols in these two sets re more distinct, i.e., less confused with the other symbols in the sme set. For exmple, if hit rtes would be the sole bsis of performnce, then Set 1 would be deemed the best set due to the higher men hit rtes of its symbols. However, since the symbols re designed to 70

71 pper simultneously s set, distinctness from the other symbols is importnt. In other words, ech symbol should be mnifest nd not esily confused with the other symbols. When the symbols were compred between sets cross ll countries (Tbles 21 to 23), Set 1 hd the lest number of confusions for only 4 of its seven symbols ( cmer, selfview, still picture, nd videophone ). Set 3 s symbols hd the lest number of confusions for document cmer nd microphone, mking them eqully suitble for these referents since their hit rtes were lso even bove ISO s 67% level. In ddition, studying instnces of flse lrms or confusions cn lso help in two wys. It cn help decide which mong symbols with similr hit rtes re better (the one with lesser flse lrms, or less like to be confused with the other symbols), nd they cn help identify different confusion ptterns between user (subject groups) nd thus different mens of solving the problems of wrong symbol ssocitions s discussed below. Confusions cn be clssified into two types: symmetric nd symmetric (Noln, 1989). The former occurs when subjects chose symbol x when presented with referent y, nd symbol y when presented with referent x. Symmetric confusions usully suggest visul or conceptul similrities. These were exemplified by the selfview / still picture (visul similrity) nd hndsfree / microphone symbols of Set 1 (conceptul similrity). Another cse of symmetric confusion due to visul similrities would be tht of the cmer / document cmer symbols of Set 3. These symmetric confusions were present in lmost ll countries. Thus, the problem my not be culturlly-linked but design problem tht the symbols concerned were visully too similr. Moreover, ccording to bsic ergonomic design principles, these symbols would then be prime considertions for re-design or replcement to mke them more distinct from the other. Asymmetric confusions occur when subjects simply chose the wrong symbol for given referent. For exmple, Tble 13 showed tht Set 1 s symbol for document cmer (1) ws wrongly selected by respondents from Indonesi when the referents cmer (5), still picture (4) nd videophone (1) ws presented. Between sets, Set 2 clerly hd the most number of confusions bove this level nd Set 1 the lest in ll seven countries. Across symbol sets, selfview nd videophone symbols hd the highest instnces of being mistken s representing other functions. More differences were noted when the countries were compred bsed on symbol confusions. For exmple, while Thilnd hd the most instnces of confusing Set 1 s symbol for cmer thn the other four countries, it hd the lowest instnces of confusing Set 1 s videophone symbol for other functions. In symmetric confusions, the problem my lie in the vgueness or too much generlity of the symbol/s in question with regrds to the other referents. Thus, they cn esily be ssocited with severl referents. The results bove showed the countries with vrying ptterns of symmetric confusions. These could be useful in deciding which symbol (or referent) in ech country my need more explntions nd tests to void vgueness nd misconceptions of the functions being represented. 71

72 Regrding the two western subject groups, gin, Set 1 hd the lest instnces of significnt confusions in both countries. However in Set 1, Swedish subjects tend to hve more problems confusing the hndsfree function s represented by the symbols for videophone (9) nd microphone (9). In turn, the Americns confused the hndsfree more with the microphone function (7) (Tbles 18 nd 19). The Swedish group showed symmetricl confusions between the selfview nd still picture symbols of Set 1 (11, 13). These were not observed with the US subjects. When confusions re concentrted to one or few referents, they lso represent the degree of confusing symbol to nother symbol presented t the sme time. For exmple, Set 1 s microphone symbol ws often ssocited with the referent hndsfree (Indonesi nd Mlysi). Such situtions could men tht the symbols were be so similr to ech other tht re-design is on order to mke them more distinct. TABLE 21. Totl number of confusions or flse lrms per country in Set 1 Countries Set 1 Ind. Ml. Phi. Thi. S.L. Swe. USA Totl Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Totl TABLE 22. Totl number of confusions or flse lrms per country in Set 2. Countries Set 2 Ind. Ml. Phi. Th. S.L. Swe. USA Totl Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Totl

73 TABLE 23. Totl number of confusions or flse lrms per country in Set 3 Countries Set 3 Ind. Ml. Phi. Th. S.L. Swe. USA Totl Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Totl Lstly, missing vlues were instnces when some of the subjects gve no response or nswers during the cued response tests. Tbles 13 to 19 illustrtes the distribution of the missing vlues cross countries for ech set. Tble 20 summrises the totl missing vlues under ech set for ech country. There were no mjor differences between countries for missing vlues in Prt One. Between sets, however, Set 2 hd the highest instnces of missing vlues. Between countries, Indonesi, Thilnd nd Sweden hd the most instnces of missing vlues mong the seven countries. Sri Lnk ws the opposite with no missing vlues in the cued response test. Furthermore, lthough Sri Lnk hd the lowest hits for some of the symbols in both tests, it hd the lowest missing vlues, especilly when compred to Indonesi nd Thilnd. This could suggest type of response bis nd/or differences in understnding test procedures not reflected in the other prmeters. When rryed with the other non-hit prmeters or errors, missing vlues re lso very importnt since they represent situtions wherein the prospective user plinly lcks the knowledge of which mong the symbols represent the desired function (referent). In ctul user scenrios, they cn be kin to non-use or underuse of the device Combining the objective performnce prmeters As discussed erlier, it would be helpful to simultneously consider the different objective prmeters to be ble to compre the videophone symbols more. Tble 24 shows how the videophone symbols performed ginst ech other (different symbols from the 3 sets for ech referent) bsed on the three objective prmeters tht could be derived from Prt Two. The best performing videophone symbols were then determined by combining these three objective prmeters (Tble 25). The results showed tht in mjority of the referents nd mong most countries, the symbols belonging to Set 1 performed best. Tht is, symbols from Set 1 were correctly ssocited to the intended function or referent, lest confused to represent nother referent, nd lest number of no nswers (possibility of not being understood). Understndbly, for some countries symbols from Set 3 were 73

74 74 eqully good (e.g., Set 3 s document cmer, hndsfree, microphone, still picture, nd videophone). This could be noted for the countries of Indonesi, Mlysi, Sri Lnk nd Sweden. TABLE 24. Videophone symbols performnces per country group bsed on the three objective prmeters in Prt Two. For ech referent, the symbol with more signifies better performnce (i.e. higher hit rtes, lower number of flse lrms, lower number of missing vlues). Indonesi Mlysi Philippines Thilnd Sri Lnk Sweden USA Referent Set Hit FA MV Hit FA MV Hit FA MV Hit FA MV Hit FA MV Hit FA MV Hit FA MV Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Legend: Ind.=Indonesi, Ml.=Mlysi, Phi.=Philippines, Thi.=Thilnd, S.L.=Sri Lnk, Swe.=Sweden, U.S.A.=United Sttes of Americ.

75 TABLE 25. Best videophone symbols when combining hit rtes, flse lrms (confusions) nd missing vlues (no nswers), per country. Numbers indicte Symbol Set. Best performing symbols for ech referent tested using the three symbol sets (Sets 1,2 nd 3) Referent Ind. Ml. Phi. Thi. S.L. Swe. USA Cmer Doc. Cmer 1 1 & & Hndsfree 1 1 & 3 1 & Microphone 1 & 3 1 & Selfview Still Picture Videophone Legend: Ind.=Indonesi, Ml.=Mlysi, Phi.=Philippines, Thi.=Thilnd, S.L.=Sri Lnk, Swe.=Sweden, U.S.A.=United Sttes of Americ Subjective certinties Subjective certinties (Tbles 26 nd 27) reflected the subjects degree of confidence or confusion with their nswers nd could thus help in weighing the hit rtes between symbols nd symbol sets. They lso helped revel possible culturl bis (observed in one country group) tht is importnt when interpreting dt wherein subjective fctors cn ply n importnt role. Dt on men subjective certinty rtings indicte how the tested symbols compre to ech other in terms of the level of confidence the subjects felt when choosing them. Confidence judgements (men subjective certinty rtings) were mostly higher in Sets 1 nd 3 compred to Set 2 in both Prts One nd Two (Tbles 26 nd 27), with those in Prt Two slightly higher thn Prt One. Confidence judgements for the symbols for microphone nd videophone were either very high (Sets 1 nd 3) or very low (Set 2) in both Prts One nd Two. The bove ptterns were quite expected. First, the symbol-referent identifiction tsk in Prt One ws generlly more difficult thn the referent-symbol ssocition tsk in Prt Two. Hence, the former tended to hve lower rtings (subjects hd lower levels of certinty in their nswers) thn the ltter. Second, Set 2 s symbols re mostly bstrct compred to the grphicl symbols of Sets 1 nd 3. This contributed further to the difficulty in identifying or mtching the trget referents with Set 2 s symbols. Between country groups, it ws lso interesting to note some differences in trends with regrds to hit rtes nd confidence judgements (subjective certinties). Swedish subjects s group tended to be more conservtive (i.e. lower men rtings) in Prt One compred to the six other country groups. Sri Lnk ws the 75

76 opposite. Although it hd the lowest men hit rtes in Prts One nd Two, it exhibited higher subjective rtings thn most of the other countries. TABLE 26. Prt One results on Confidence Judgements (Men Subjective Certinty Rtings) per country (n = 48 per country, except USA, where n = 40) SET 1 Ind. Ml. Phi. Th. S.L. Swe. USA. Mens [1] Cmer [4] Doc. Cmer [7] Hndsfree [10] Microphone [13] Selfview [16] Still Picture [19] Videophone Mens SET 2 [2] Cmer [5] Doc. Cmer [8] Hndsfree [11] Microphone [14] Selfview [17] Still Picture [20] Videophone Mens SET 3 [3] Cmer [6] Doc. Cmer [9] Hndsfree [12] Microphone [15] Selfview [18] Still Picture [21] Videophone Mens Legend: Ind.=Indonesi, Ml.=Mlysi, Phi.=Philippines, Th.=Thilnd, S.L.=Sri Lnk, Swe.=Sweden, U.S.A.=United Sttes of Americ. 76

77 TABLE 27. Prt Two results on Confidence Judgements (Men Subjective Certinty Rtings) per country (n = 48 per country, except USA, where n = 40) SET 1 Ind. Ml. Phi Th. S.L. Swe. U.S.A. Mens [1] Cmer [4] Doc. Cmer [7] Hndsfree [10] Microphone [13] Selfview [16] Still Picture [19] Videophone Mens SET 2 [2] Cmer [5] Doc. Cmer [8] Hndsfree [11] Microphone [14] Selfview [17] Still Picture [20] Videophone Mens SET 3 [3] Cmer [6] Doc. Cmer [9] Hndsfree [12] Microphone [15] Selfview [18] Still Picture [21] Videophone Mens Legend: Ind.=Indonesi, Ml.=Mlysi, Phi.=Philippines, Th.=Thilnd, S.L.=Sri Lnk, Swe.=Sweden, U.S.A.=United Sttes of Americ Symbol nd set preferences Preferences generlly indicte the esthetics; the subjects personl judgement tht symbols nd set best represented the referents in question (Böcker, 1993 nd Mckett-Stout nd Dewr, 1981). Aesthetic judgements s reflected in preference tests cn be helpful when confronted with choosing between symbols representing the sme function whose performnce prmeters were nerly equl. Regrding preferences (Tbles 28 nd 29), t lest 50 percent of the subjects preferred five of the seven symbols of Set 1 compred to those of symbols of Sets 2 nd 3. Between sets, Set 1 ws clerly preferred mong the three sets (58.3% to 68.8%) mong the Asin countries. However, regrding set preferences, the hlf of the US subjects, preferred Set 3 nd 45% preferred Set 1. Hlf of the Swedish 77

78 subjects preferred Set 1 nd lmost 40% preferred Set 3. Thus, big portion of the western subjects seemed to prefer Set 3 side from Set 1. TABLE 28. Symbol preferences in (%) with ll countries combined (N = 240). TABLE 29. Symbol set preferences in (%) per country. Symbol Preferences Set Preferences Set 1 Set 2 Set 3 Set 1 Set 2 Set 3 Cmer Indonesi Doc. Cmer Mlysi Hndsfree Philippines Microphone Thilnd Selfview Sri Lnk Still Picture Sweden Videophone U.S.A Mens Semntic Differentil Tests Tble 30 summrises the results of the semntic differentil tests for the three symbol sets. Generlly, symbols representing fmilir or common functions hd high semntic rtings (ex. cmer nd microphone) cross ll sets. Among the sets, Set 2 hd the lower semntic rtings. Set 2 s low hit rtes nd semntic scores confirmed one of the hypotheses; tht is, bstrct symbols would tend to hve poor comprehension nd rting. Novel or complex functions in turn hd lower semntic rtings (ex. hndsfree, selfview) nd could indicte gin the need for more lerning time. Furthermore, frequency distributions of the rtings cn lso help nlyse the reltionships between the functions nd the symbols representing them. Figures (Appendix 3) contin the precent distribution for the 21 symbols depicting the seven-videophone functions. Positive skewness (high frequency distributions towrds positive side, i.e. rtings of 5 to 7) were chrcteristic for symbols with high hits (t lest 67%) nd certinty rtings in both Prt One nd Prt Two. For exmple, Sets 1 nd 3 s symbols for microphone nd videophone hd hit rtes of sixty percent or more in both Prts One nd Two. Correspondingly, their SDT scores were highly skewed on the positive side. These strongly suggest the subjects high levels of confidence nd understnding of these symbols' intended menings. The symbols cn thus be expected to perform stisfctorily when used by the subjects. On the other hnd, negtively skewed nd pltykuric (low humped or scttered) distributions cn suggest lower levels of semntic or connottive concepts of the symbols in reltion to the functions. These cn be seen from the frequency distributions of symbols for hndsfree (Sets 1-3), microphone (Set 2), 78

79 still picture (Sets 2-3) nd videophone (Set 2). These suggest the need to re-design the symbols. On the other hnd, the referents or functions themselves my lso need to be lerned or explined further. This ltter view in wy lso gins support by the observtions mde by Rutter nd Beck (1997). In usbility testing of cordless telephones, their subjects were tested on their comprehension of lbels (functions) currently vilble in such telephones. Very few of these functions were understood by the mjority of their subjects (Rutter nd Beck, 1997). Thus, when confronted with using new devices, prt of the problem my lie on possibility tht the subjects do not even understnd some of the functions themselves. TABLE 30. Men rtings of ech semntic scle, per set, per country (n = 48 per country, except U.S.A. where n = 40). Semntic Scles Set 1 Ind. Ml. Phi. Th. S.L. Swe. USA Meningful Concrete Fmilir Simple Cler Set 2 Mening Concrete Fmilir Simple Cler Set 3 Mening Concrete Fmilir Simple Cler Regrding the western subjects, gin, the Swedish group s whole gve lower men semntic rtings compred to the Americn group (Tble 17). Although, there is high correltion between the hit rtes nd ll the semntic scles, both the Swedish nd US subjects hd highest rtings for the symbols meningfulness nd simplicity, with the ltter group putting lso prime importnce to concreteness The Finnish Elderly (Pper 5) The multiple index tests used in the Asin, Swedish nd US studies were similrly crried out with 29 elderly subjects (14 women nd 15 men). The women were 61 to 81 yers old (men ge: 69.6 yers) nd the men 60 to 88 yers old (men ge: 70.9 yers). The men ge for the whole group ws 70.3 yers. Compred to the subjects from Southest Asi nd even the USA, the Finnish elderly filed to do 79

80 prts One nd Three. Mjority of the subjects climed tht the tsks were either too difficult or too complicted. Nonetheless, Tbles 31 nd 32 contined the results for Prt Two while Tbles 32 nd 33 contin the results for Prt Four. Compred to younger subjects (from Asi nd US), the elderly subjects hd much lower hit rtes in ll symbols of the 3 sets used. In Prt Two, the subjects lso expressed the level of certinty to their nswers using response scle from "1" (very certin) to "7" (very uncertin). Tble 32 shows the computed men certinty level nd stndrd devition for ech icon in different icon fmily. TABLE 31. The results for cued-response tests (Prt Two), Finnish elderly s subjects, N=29. Set 1 Set 2 Set 3 Referents n % n % n % Cmer Document cmer Hndsfree Microphone Selfview Still Picture Videophone TABLE 32. Certinty rtings for the cued-response tests (Prt Two) using Finnish elderly, N=29. Set 1 Set 2 Set 3 Referents Men S. D. Men S. D. Men S. D. Cmer Document cmer Hndsfree Microphone Selfview Still Picture Videophone

81 TABLE 33. The results for Prt Four using Finnish elderly subjects, N=29. Referents Set 1 % Set 2 % Set 3 % None % Cmer Document cmer Hndsfree Microphone Selfview Still Picture Videophone Fmily In Prt Four, the subjects chose for ech of the seven videophone referents the symbol or icon tht he/she preferred most. The subjects lso hd to choose from the three icon fmilies the icon fmily he/she preferred most. Tble 33 presents the number nd percentges of the subjects, who liked the icon best nd the number of subjects who did not nswer to the question. It ws quite evident tht the elderly subjects preferred the most of the symbols of Set 3 nd even preferred Set 3 s whole. These were in strk contrst to subjects from Southest Asi nd USA, who preferred Set 1 nd its symbols, the proposed stndrd set by ETSI. Prt Four lso contined questions regrding ttitudes towrds technology. The results showed tht mjority of the subjects, both men nd women greed either totlly or lmost totlly to the view tht the dvntges of technologicl progress outweigh the disdvntges (Tble 34). 91% of the men nd 54 % of the women thought tht the progress in technology totlly or lmost totlly mkes life lot esier (Tble 35). The progress in technology ws thought to mke life more difficult by bout 13 % of the subjects. TABLE 34. Frequency distribution for the question: Advntges of technologicl progress outweigh the disdvntges. N % Men % Women % Do not gree t ll Totlly gree However, bout sixty-seven percent (67%) of the subjects (54 % of the men nd 77 % of the women) were totlly or lmost totlly worried bout some spects of tody s technologicl progress (Tble 36). Eighteen percent of the men were not 81

82 worried t ll. These findings showed tht elderly people re not entirely hostile to modern technology s commonly believed. Wht my worry them, more probbly, is the usbility of modern products, so life cn indeed be better or esier for them. TABLE 35. Frequency distribution for the question: Progress in technology mkes life lot esier: N % Men % Women % Do not gree t ll Totlly gree TABLE 36. Frequency distribution for the question: Some spects of tody s technologicl progress re worrying. N % Men % Women % Do not gree t ll Totlly gree The trend of demogrphic evolution shows tht the Europen popultion, in prticulr, is one tht is becoming old. This trend will hve implictions on the socil structure, where respect for n independent living should be tken into considertion, tking into ccount the new services nd products just vilble or those under development, such s tele-lrms nd teleservices to ssist people t home or elsewhere. There cn be mny plesures nd stisfction to seniority, but there re lso negtive physicl, cognitive, nd socil consequences of geing. Understnding the humn fctors of geing cn led to computer designs tht will fcilitte ccess by the elderly. The benefits to the elderly include prcticl needs for writing, ccounting, nd the full rnge of computer tools, plus the stisfctions of eduction, entertinment, socil interction, communiction nd chllenge. In this cse, symbols were intended to improve the usbility of product (i.e. videophone). However, the results showed tht the opposite might occur mong the elderly. Other benefits include incresed ccess of the society to the elderly for their experience, incresed prticiption of the elderly in society through communiction networks, nd improved chnces for productive employment of the elderly. (Schneidermn, 1992). The results in this cse study emphsise these 82

83 points s well. Only by conscious nd ctive prticiption of the elderly in designing nd developing modern products will their needs be truly met Generl discussion The results of the tests in the different countries showed tht the spontneous identifiction test (Prt One) hd significntly lower hit rtes compred with the hit rtes in the cued response test (Prt Two). Prt One nd together with Prt Two simulte (though in limited mnner) rel videophone cll scenrios wherein the symbols re plced on the terminl with one function to be performed t time. Prt One is sitution kin to using the technology the first time with miniml knowledge nd instructions. Thus, it ws not surprising tht only fmilir symbols or those tht re concrete representtions of the functions would hve higher hit rtes (ex. microphone, nd cmer), confirming one of the min hypotheses of this study. Further, it ws interesting to note tht symbols could indeed be quite difficult to understnd or comprehend bsed on the very low hit rtes in Prt One. The cued-response tests were similr to situtions when using the videophone the first few times but with more informtion nd instructions given. These findings give further credence to the importnce of providing the prospective users ll the possible cues nd lerning ids when ttempting to use new product or device the first few times (Lund, 1997, Shhnvz, 1998 nd Wolff nd Woglter, 1998). By using the cued-response tests severl objective prmeters were derived tht helped revel other possible differences between the cndidte symbols nd between subject groups. These cn be summrised s follows: ) Overll, the min studies showed tht hit rtes or correct symbol-referent ssocitions re still the most importnt objective prmeter of symbol performnce. In this spect, Set 1 s symbols performed best. Six of its 7 symbols were correctly nd esily ssocited to their intended functions or referents for ll subject groups. b) However, hit rtes re not the only importnt prmeter, especilly when evluting severl sets of symbols nd using different subject groups. Non-hit prmeters such s flse lrms or confusions nd missing vlues cn be of gret importnce s well. For exmple, Set 3 s symbols fred comprtively well in being ssocited to the intended functions, ttining hits s high s (if not higher thn) Set 1. In these cses, the non-hit prmeters such s the flse lrms nd missing vlues were very helpful in determining which symbol performed better mong the subject groups. 83

84 c) Flse lrms or confusing the symbol s representing the wrong referent is very good guge of studying how distinct symbol is compred to the other symbols presented t the sme time. This prmeter becomes more useful when used mong subject groups with different culturl bckgrounds (i.e. countries). The predominnt types of confusions (symmetricl nd symmetricl) in ech group could be used to study further how the symbols re understood nd even confused by the subjects. These would then be helpful to determine if the problems would merit () re-design of the symbols; (b) replcement of the symbols; (c) more explntion or emphsis on how the definition of the functions. In short, would the recommendtions focus on symbol re-design or replcements, or on lerning or instructionl ids to mke sure tht the device nd its functions re well understood? d) Missing vlues re lso very importnt since they represent situtions wherein the prospective user plinly lcks the knowledge of which mong the symbols represent the desired function (referent). In ctul user scenrios, they cn be kin to non-use or under-use of the device. If this would be one of the min problems, recommendtions would be more on steps ensuring fmilirity with the device nd its functions. In turn, the subjective prmeters such s subjective certinties, semntic scles nd preferences were lso helpful in gining insight in subject group differences especilly to certin possible subjective bises tht my be culture-relted. They lso showed tht when intended for diverse user groups, grphicl symbols tht re more concrete nd fmlir hve better chnces of being understood correctly (ssocited to the correct functions). The bove results lso merited further discussion on the current issues of using spontneous identifiction nd cued-response tests when testing symbols, which ws ctully one of the minor objectives of the study. Wolff nd Woglter (1998) contended tht the cognitive processes tht people usully perform when encountering grphicl symbols in the rel world re more closely mirrored by the processes involved in open-ended tests thn cued-response tests. The evlution of symbols in open-ended tests (like the spontneous identifiction test) hs lso been regrded s less-likely to produce constrined nd distorted nswers compred to the cued-response type of tests (Neisser, 1987). These contentions re vlid nd will hold when stnd-lone symbols re the ones in question (ex. trffic signs nd symbols, rod signs, nd most directionl or informtionl symbols in public plces nd work sites). But when symbols re designed to pper together or s sets, results of open-ended tests cn be quite difficult s experienced in the studies. Although, they were helpful in reveling the relity tht symbols could still be very poorly recognised or understood by lrge number of people, eliciting cler-cut informtion could be difficult. Criteri need to be estblished for the kinds of nswers tht would be counted s correct. This ws one of the relistions in the 84

85 study tht there could lwys be some subjective judgement of correctness of the prticipnts responses. This is even mgnified considering the different lnguges or mother tongues of the subject groups tested. Thus, relibility ssessment would be better if there would be more thn one judge something tht ws lso pointed out by Dewr (1994) nd Wolff nd Woglter (1998). Understndbly, the downside then would be the extr time nd resources needed when more judges re needed. Further, the study lso reveled the difficulty of performing openended tests with elderly subjects. This ppered to be difficult tsk for this subject group compred to the cued-response test. The cued-response tests s dopted from Böcker (1993) hd more strightforwrd mnner of quntifying correct responses s well s the errors. Together with the mnner of presenting ech referent nd one symbol set t time, they more closely mimic the projected user-scenrio of how the videophone symbols would be used in rel life tht is, situtions wherein the users hve some knowledge of the device t hnd nd some intended functions to be represented by the symbols. On the other hnd, spontneous identifiction tests, when properly done, simulte first-time use scenrios, wherein both device nd symbols cn be quite new nd unfmilir. Correct symbol-referent ssocitions (hit rtes) for ech test type cn thus be quite different but hve their own uses nd relevnce in symbol evlution. Lstly, in this reserch, the different tests were designed to mesure initil ( priori) mening nd ppropriteness, ese of lerning nd remembering, s well s the probbility of confusion with other designs. The methods utilised were lrgely bsed on the multiple index pproch (MIA) developed by Böcker with the ETSI (Böcker, 1993) nd with some modifictions. An open-ended test (spontneous identifiction) nd semntic differentil scles were lso employed. The ltter ws in considertion of the recommendtion tht it remins s n importnt method of mesuring symbol understnding. The issue between using open-ended nd cuedresponse tests ws lredy discussed in detil erlier. Nonetheless, when Böcker developed the MIA nd performed the tests, one of the min objectives ws to decide which mong severl cndidte grphicl symbols would best represent seven prticulr videophone functions. The principles nd logic of the MIA were discussed together with the results they obtined. In turn, present reserch cknowledges the ppropriteness of the MIA s the method when testing grphicl symbols intended to pper together in n interfce. Using this pproch of multiple test prmeters helped not only in identifying best performing symbols, tht is, which mong the different symbols were understood correctly s they were intended to be. They lso ided in studying how different subject groups (country groups) could differ in their understnding of symbols nd how such differences could help in determining the interventions needed to mke grphicl symbols better interfces. 85

86 7 CONCLUSIONS AND RECOMMENDATIONS Bsed on ll the studies, the ims of the project were relised. Different tests were utilised in order to evlute cndidte telecommuniction symbols developed in the west using subjects from estern (Asin) nd western countries. A qusiexperimentl pproch utilising multiple test prmeters ws dopted in testing different sets of grphicl symbols for the videophone (icons from CIAJ of Jpn, IDEO of London, both from studies by Tudor (1994), nd ETSI of Europe). The following conclusions could then be drwn. First, mjority of the icons tested ws poorly recognised by ll subject groups especilly when bsed on the ISO % recognition rte on spontneous identifiction tests s bsis. The icons or symbols bove this level were mostly concrete representtions of their referents nd/or quite fmilir to the users lredy (CIAJ nd ETSI symbols). Second, lthough Asin subjects generlly hd comprble recognition rtes to western subjects, they hd more instnces of confusions. Third, empiricl tests using multiple indices re importnt to properly evlute the symbols on their usbility especilly cross different user groups. The evlution method should reflect expected usge scenrios to determine the suitbility of ech symbol lone, nd together with the other symbols s well. Hit rte remins n importnt prmeter, but how it will be derived hs to be considered closely. For exmple, cued responses nd spontneous identifiction re useful to determine if the symbols indeed re understood with nd without/or lest of cues respectively. It must be remembered though, tht when using open-ended tests such s the spontneous identifiction test, the method on how to ccurtely interpret nd judge correct nswers hve to be crefully considered. At the sme time hits rtes lone re not enough. Flse lrms, missing vlues, subjective certinties nd preferences cn lso be very useful fctors in order to mke deeper nlyses. They enble the tester to see other often subtle but importnt differences (i.e. subjective bises, confusing symbols to other referents nd symbols) on how users perceive nd understnd symbols. Using semntic differentil scles could lso help in understnding how symbols re perceived in reltion to the functions they re supposed to represent. Tken ltogether, they re highly beneficil not only in properly interpreting test results of different user groups, but lso in formulting instructions nd other ids in lerning to use new products fster nd more stisfctorily. Modern society hs come bck to use grphicl symbols, especilly s n essentil prt of the interfce between humns nd their incresingly complex mchines. Specificlly, they re primry GUI component, helping put order nd sense to the complexities of the humn-computer systems so bundnt nowdys. They offer numerous dvntges over text-bsed interfces, but with their own disdvntges of course, both of which need to be weighed when contemplting the use of symbols. However, s discussed bove, designing symbols tht cn be 86

87 understood by mjority of the trget users is not simple tsk. Prt of the complexity is how people perceive symbols nd render mening to it. Understnding something bout the humn visul imge processing cn ssist in gining few insights of how they re understood. But knowledge nd wreness of the possible different contexts in which symbols cn be used or perceived, nd the different levels of user experience nd expecttions cn eqully (nd gretly) ffect how the symbols in question re perceived s they re intended to be. These issues hve led us to the criticl role plyed by symbol testing nd evlution. Severl methods re vilble depending on one's objective, but in lmost ll cses user involvement is of prime considertion. Evluting grphicl symbols is complicted nd the use of multiple indices cn be of gret help in understnding how symbols re recognised, understood nd lerned by different user groups. Still, there re few limittions in this study tht need to be ddressed nd re thus recommended towrds better designing nd testing of symbols: ) Since the studies were minly bsed on pencil nd pper tsks, more insights my be obtined if: - the symbols re tested using ctul interfces (computer screens, dilpds, etc.) nd under more vried scenrios (e.g. with nd without help options, norml vs. highly-stressed situtions, cross-pplictions) - the functions themselves need to be tested for their comprehension mong the subjects or prospective users b) The semntic differentil testing results were interesting but discussed here in still quite limited mnner nd needs deeper nlyses. c) In more ergonomics pproch, users nd experts should both be considered in ll possible stges of design nd testing. One simple but concrete follow-up is to design n itertive type of method to generte grphicl symbols. Briefly, such method will involve process of eliciting from the subjects their own ides of how the symbols should look like, identifying the design fctors behind them tht cn then be used by ergonomics experts nd designers to contrive lterntive symbols. d) An often-expressed issue in grphicl symbols use is the pplicbility nd potentil benefit of using the sme grphicl symbols for the sme functions cross different devices or equipment. It is highly recommended to do more studies to vlidte this view not only cross vrying devices but lso cross vrying country or culturl groups. 87

88 8 REFERENCES Abeyseker, J.D.A., Shhnvz, H., & Chpmn, L.J. (1990). Ergonomics in developing countries. In Advnces in Industril Ergonomics nd Sfety II (pp ). London: Tylor nd Frncis. Americn Ntionl Stndrds Institute (ANSI). (1993). Accredited stndrd on sfety colors, signs symbols, lbels nd tgs, Z Wshington DC: Ntionl Electricl Mnufcturers Assocition. Brelsford, J.W., Woglter, M.S., & Scoggins, J.A. (1994). Enhncing comprehension nd retention of sfety-relted pictorils. In Proceedings of the Humn Fctors Ergonomics Society 38th Annul Meeting (pp ). Böcker, M. (1993). A multiple index pproch for the evlution of pictogrms. In: Proceedings of the 14th Interntionl Symposium on Humn Fctors in Telecommunictions (pp ). Drmstdt, Germny, My 11-14, Heidelberg: R. v. Decker s Verlg, G. Schenck GmbH. Böcker, M. (1997). Symbol design, In: A Focus Seminr in Symbol Design nd Evlution: Forum for Corporte Usbility within Ericsson. Stockholm: Ericsson AB. Cron, J.P., Jmieson, D.G., & Dewr, R.E. (1980). Evluting pictogrphs using semntic differentil nd clssifiction techniques. Ergonomics, 23, Clrke, A.M., & Brown, S. (1992). Pictogrms in multipoint videotelephony: n interntionl study. In E.J. Lovesey (Ed.), Contemporry Ergonomics 1992 (pp ). London: Tylor nd Frncis. Collins, B.L., & Lerner, N.D. (1982). Assessment of fire sfety symbols. Humn Fctors, 24(1), Dvies, S., Hines, H., Norris, B., & Wilson, J. (1998). Sfety pictogrms: re they getting the messge cross? Applied Ergonomics, 29(1), Dewr, R. (1994). Design nd evlution of grphic symbols. In Proceedings of Public Grphics (pp ). Utrecht, Netherlnds: University of Utrecht, Deprtment of Psychonomics. Dewr, R. (1999). Design nd evlution of public informtion symbols. In H.J.G. Zwg, T. Boersem, & H.C.M. Hoonhout (Eds.), Visul Informtion for Everydy Use - Design nd Reserch Perspectives (pp ). London: Tylor nd Frncis. Dewr, R.E., & Ells, J.G. (1979). The semntic differentil s n index of trffic sign perception nd comprehension, Humn Fctors, 19(2), Dreyfuss, H. (1972). Symbol sourcebook. New York: McGrw-Hill. 88

89 Esterby, R.S., & Hkiel, S.R. (1981). Field testing of consumer sfety signs: the comprehension of pictorilly presented messges, Applied Ergonomics, 12(3), Edwrds, A.L., & Porter, B.C. (1972). The ffective domin. Wshington DC: Ntionl Specil Medi Institutes-Gryphon House. Edworthy, J., & Adms, A. (1996). Wrning design - A reserch prospective. London: Tylor nd Frncis. Encrt Encyclopedi, (1997). Asi. Microsoft TM Encrt 1997 Encyclopedi CD Version. Encrt Encyclopedi, (1997). Technology. Microsoft TM Encrt 1997 Encyclopedi CD Version. Encrt Encyclopedi, (1997b). Microprocessor. Microsoft TM Encrt 1997 Encyclopedi CD Version. Europen Telecommunictions Stndrds Institute (ETSI). (1993). Technicl Report - Humn Fctors (HF), Results of n evlution study of pictogrms for point-to-point videotelephony, Sophi Antipolis Cedex: ETSI. Eysenck, M.W., & Kene, M.T. (1999). Cognitive Psychology (3rd ed.). Est Sussex: Psychology Press. Fmilnt, M.E., & Detwiler, M.C. (1993). Iconic reference: evolving perspectives nd orgnizing frmework. Interntionl Journl of Mn-Mchine Studies, 38, Gittins, D. (1986). Icon-bsed humn computer interction. Interntionl Journl of Mn-Mchine Studies, 24, Green, P., & Pew, R.W. (1978). Evluting pictogrphic symbols: n utomotive ppliction. Humn Fctors, 20, Hllidy, M.A.K. (1970). Lnguge structure nd lnguge function. In J. Lyons (Ed.), New Horizons in Linguistics. Middlesex: Penguin. Henerson, M.E., Morris, L.L., & Fitz-Gibbon, C.T. (1987). How to Mesure Attitudes. Newbury Prk: Sge Publictions. Horton, W. (1994). The Icon Book - Visul symbols for computer systems nd documenttion. New York: Wiley nd Sons. Interntionl Telecommunictions Union (ITU). (1995). Recommendtion F.910 (02/95) - Procedures for designing, evluting nd selecting symbols, pictogrms nd icons. Genev: ITU. Jones, S. (1983). Stereotipy in pictogrms of bstrct concepts. Ergonomics, 26(6), Keesing, R.M. (1974). Theories of culture. Annul Review of Anthropology, 3,

90 Kline, T.J.B. Ghli, L.M., Kline, D.W., & Brown, S. (1990). Visibility distnce of highwy signs mong young, middle-ged, nd older observers: Icons re better thn text, Humn Fctors, 32, Kroemer, K. (1997). Fitting the Tsk to the Humn (5th ed.). London: Tylor nd Frncis, ISBN Lin, R. (1992). An ppliction of the semntic differentil to icon design, In Proceedings of the Humn Fctors Society 36th Annul Meeting, Atlnt, Georgi, October 12-16, 1992, Vol. 1, (pp ). Snt Monic: Humn Fctors Society. Lin, R., & Kreifeldt, J.G. (1992). Understnding the imge functions for icon design, In: Innovtions for Interctions. In Proceedings of the Humn Fctors Society 36th Annul Meeting, Atlnt, Georgi, October 12-16, 1992, Vol. 1, (pp ). Snt Monic: Humn Fctors Society. Lin, R., Kreifeldt, J.G., & Chi, C.F. (1992). A study of evlution design sufficiency for iconic interfce design: the design perspective. In Lovesey, E.J. (Ed.), Contemporry Ergonomics 1992, (pp ). London: Tylor nd Frncis. Lodding, K.N. (1982). Iconic interfcing. IEEE Computer Grphics nd Applictions, 3, Lund, A.M.L., ( Expert rtings of usbility mxims. Ergonomics in Design, 5(3), Mckett-Stout, J., & Dewr, R. (1981). Evlution of symbolic public informtion signs. In Proceedings of Humn Fctors Society Conference, Vol. 23, No. 2, (pp ). Mguire, M.C. (1985). A review of humn fctors guidelines nd techniques for the design of grphicl humn-computer interfces. Computers nd Grphics, 9(3), Mgyr, R.L. (1990). Assessing icon ppropriteness nd icon discriminbility with pired comprison testing procedure, In Countdown to the 21st Century. Proceedings of the Humn Fctors Society 34th Annul Meeting, Orlndo, Florid, October 8-12, 1990, Vol. 2, (pp ). Snt Monic: Humn Fctors Society. My, J., Böcker, M., Brnrd, P.J., & Green, A.J.K. (1990). Chrcterising structurl nd dynmic spects of the interprettion of visul interfce objects, In Proceedings of Esprit '90. Dordrecht: Kluwer Acdemic Publishers. Modley, R. (1966). Grphic symbols for world-wide communiction. In Kepes, G. (Ed.), Sign, Imge nd Symbol, (pp ). London: Studio Vist. Neisser, U. (1987). The present nd the pst. In Second Interntionl Conference on the Prcticl Aspects of Memory. Swnse, Wles. 90

91 Noln, P.R. (1989). Designing screen icons: rnking nd mtching studies. In Proceedings of the Humn Fctors Society 33rd Annul Meeting, (pp ), Snt Monic: Humn Fctors. Olmsted, W.T. (1999). The usbility of symbols for helth cre fcilities: the effects of culture, gender nd ge. In Zwg, H.J.G., Boersem, T., & Hoonhout, H.C.M. (Eds.), Visul Informtion for Everydy Use - Design nd Reserch Perspectives, (pp ). London: Tylor nd Frncis. Oppenheim, A.N. (1983). Questionnire Design nd Attitude Mesurement. London: Heinemnn. Osgood, C.E., Suci, G., & Tnnenbum, P. (1957). The Mesurement of Mening. Urbn: University of Illinois Press. Ossner, J. (1990). Trnsntionl symbols - The rule of pictogrms nd models in the lerning process. In Nielsen, J. (ed.), Designing Interfces For Interntionl Use, (pp ), Amsterdm: Elsevier. Plocher, T.A., Grg, C., & Chestnut, J. (1999). Connecting culture, user chrcteristics nd user interfce design, In Bullinger, H. & Ziegler, J. (Eds.), Advnces in Humn Fctors/Ergonomics, Proceedings of the 1999 Humn-Computer Interction Conference: Ergonomics nd User Interfces, (pp ). München, Germny, August 22-26, Rogers, Y. (1986). Evluting the meningfulness of icon sets to represent commnd opertions, In Proceedings of Humn Computer Interction 86, (pp ). London: British Computer Society Ed. Rutter, B.G., & Beck, A.M. (1997). Evolution of cordless phone. Ergonomics in Design, 5(3), Slsoo, A. (1990). Towrds usble icon sets: cse study from telecommunictions engineering, In Countdown to the 21st Century. Proceedings of the Humn Fctors Society 34th Annul Meeting, Orlndo, Florid, October 8-12, 1990, Vol. 1, (pp ). Snt Monic: Humn Fctors Society. Snders, A.F. (1998). Informtion processing nd design. In Chpter 29: Ergonomics, Work System Design, Encyclopedi of Occuptionl Helth nd Sfety, (pp ). Genev: ILO. Senner, W.M. (1989). The Origins of Writing. Lincoln: University of Nebrsk Press. Shhnvz, H. (1991). Trnsfer of Technology to Industrilly Developing Countries nd Humn Fctors Considertions, Center for Ergonomics of Developing Countries, Luleå: Luleå University Press, Sweden. Shhnvz, H. (1998). Culturl differences, In: Chpter 29: Ergonomics, Designing for Everyone, Encyclopedi of Occuptionl Helth nd Sfety, (pp ). Genev: ILO. 91

92 Shneidermn, B. (1992). Designing the User Interfce. New York: Addison-Wesley Publishing Compny. Stmmers, R.B. (1990). Judged ppropriteness of icons s predictor of identifiction performnce, In Lovesey, E.J. (Ed.), Contemporry Ergonomics 1990, (pp ). London: Tylor nd Frncis. Strmler, J.H. Jr. (1993). The Dictionry for Humn Fctors/Ergonomics. London: CRC Press. Sukviriy, P., & Morn, L. (1990). User interfce for Asi. In Nielsen, J. (Ed.), Designing Interfces For Interntionl Use, (pp ). Amsterdm: Elsevier. Tudor, L. (1994). The genertion nd testing of cndidte telecommunictions icons, Interntionl Conference in Humn-Computer Interction Proceedings, Vol. 11, St. Petersburg, Russi, August 2-6, (pp ). Vn Dm, (1984). Computer softwre for grphics, Scientific Americn, 251, Vor, P., Helnder, M., Swede, H., & Wilson, J., (1991). Developing guidelines for symbol design: comprison of methodologies, In Proceedings of Interfce 1991, (pp. 6-11). Webb, J.M., Sorenson, P.F., & Lyons, N.P. (1989). An empiricl pproch to the evlution of icons, SIGCHI Bulletin, 21(1), Wolff, J.S., & Woglter, M.S., (1998). Comprehension of pictoril symbols: effects of context nd test method, Humn Fctors, June 1998, Wood, W.T., & Wood, S.K. (1987). Icons in Everydy Life, In Socil Ergonomics nd Stress Aspects of Work with Computers, (pp ). Zwg, H.L.G. (1989). Predicting the public s comprehension of grphic symbols, In Megw, E.D. (Ed.), Contemporry Ergonomics 1989, (pp ). London: Tylor nd Frncis. 92

93 PAPERS PRODUCED FROM THE STUDIES 93

94 94

95 Edited by Dr. Lim Kee Yong Lind Hermn Dr. Ying K. Leung Dr. Jckie Moyes PAPER 1 A User-bsed Evlution of Telecommuniction Icons: Some Cross-Culturl Issues

96

97 A User-bsed Evlution of Telecommuniction Icons: Some Cross-Culturl Issues Pul Pimonte, M.D.; John D.A. Abeyseker, Ph.D.; Kjell Ohlsson, Ph.D. Industril Ergonomics Division Dept. of Humn Work Sciences, Luleå University Luleå, SWEDEN S , Abstrct Telecommuniction nd computer icons re often designed nd developed in western countries. These my crete problems mong potentil users in estern countries. Simple nd relible methods re needed when such icons re used in Third World countries. This pper presents pilot study utilising semntic differentils in evluting cndidte icons designed in western countries using subjects from n Asin country. The dt indicte tht icon recognition nd perception my vry in n Asin setting. Semntic differentil scores correlted highly with the evlution results nd my be useful in similr crossculturl studies of symbols s interfce tools. Keywords Icons, culture, semntic differentil testing, informtion technology, user-bsed evlution. 1. Introduction Icons cn be defined s symbols, glyphs or pictogrphs used to present concept, ide or object, (Wood nd Wood, 1987). In humn-computer interfces, they help introduce the technology (product) nd its functions. They cn speed up ccess to desired opertions, nd promote sense of use. Under the present economic globliztion wherein the designers cn be fr removed nd different from the potentil users, the opposite effects cn occur when the icons re unfmilir, difficult to understnd, incomprehensible, or unppeling. Of prticulr interest is the Asi-Pcific region. Driven by n ever-improving economy, there is concomitnt increse in the ccess nd use of computer nd other telecom products. Reserches re needed to design nd develop simple, inexpensive but relible methods to evlute icons pplicble in such countries of vstly different culturl bckgrounds. 145

98 Numerous methods hve lredy been tried in evluting icons nd other grphic designs (Lin nd Kreifeldt, 1992). The semntic differentil (SD) test developed by Osgood, Suci nd Tnnenbum (1957) ws method of mesuring the psychologicl mening of concepts or objects by direct rtings of the stimuli using scles nchored on the extremes by bipolr djectives. Kerlinger (1973) cited severl studies done on the menings of objects, personlities, pictures nd bstrct concepts The use of SD in evluting grphic symbols hve been crried out s well (Lin nd Kreifeldt, 1992). Icon chrcteristics such s concreteness, fmilirity nd lernbility were discussed. However, side from the methodology, the use of ctul potentil users s subjects is eqully importnt in studying the possible culturl implictions of using symbols such s icons in the interfces. Tudor (1994) emphsised the importnce of explining nd testing the icons in mnners reflective not only of typicl user scenrios but lso of the users themselves. In the Asin setting most, if not ll, of the icons for use in computerised products re designed in industrilised countries but trgeted for use by ntionls from vrying cultures. This pilot study ims to evlute icons designed in Jpn nd Europe (industrilised countries) using the semntic differentil pproch with subjects coming from the Philippines (n industrilly developing country). 2. Methodology 2.1 Subjects nd Mterils Thirty subjects (15 mles nd 15 femles) ged 22 to 45 yers from Metro-Mnil were rndomly selected to prticipte. All were office employees with 2 to 5 yers experience working with computers. The 23 icon referents nd the 69 icons used by Tudor (1994) were selected. The icons were grouped into three types: bstrct, concrete nd proposed bsed on their generl ttributes (Tble 1). Sixty-two of the icons were designed in Englnd. The remining 7 icons were designed in Jpn (Tble 1, Proposed Icons = Volume, Achieve Dil Tone, Hold, Music on Hold, Mute, Store, nd Spekerphone ). The questionnire hd two prts: recognition/subjective certinty tests, nd semntic differentil. Questionnires were produced per icon fmily with the icons nd their referents presented in rndom orders. The semntic differentil ws composed of 6 bipolr djectives in 7 point scles lso rrnged in rndom orders for ech of the icons or referents. The first pge contined stndrd set of instructions. 2.2 Procedure After presenting the study s objectives nd explining the icons, the subjects first underwent the recognition nd certinty tests. Icons nd referents were rndomly presented nd to be mtched correctly. For ech recognition test, certinty levels on the nswers were indicted by using 7 point scle. After one week, the 146

99 semntic differentil testing ws performed. The icons were gin presented. This time, they were to rte ech icon bsed on the 6 bipolr djectives. The sme tsks were done for the three icon fmilies. Thus, ech subject performed 3 recognition/subjective certinty tests, nd 3 semntic differentil tests, one for ech icon fmily. 3. Results nd Discussion Tbles 1 nd 2 give the summry of the results of recognition/certinty tests nd the semntic differentil testing. For the 23 referents represented by 23 icon fmilies, only 5 of the 23 referents hd men recognition percentge of t lest ABSTRACT ICONS CONCRETE ICONS PROPOSED ICONS % Men Men % Men Men % Men Men REFERENTS Recognition Rtings Recognition Rtings Recognition Rtings 1. Achieve Dil Tone 27.6% % % Answer Ringing Cll 10.0% % % Cll Log 50.0% % % Conference 80.0% % % Dilpd 83.3% % % Drop 37.9% % % Help Specific 21.4% % % Help System 36.7% % % HFAI 25.0% % % Hold 35.7% % % Messge 36.7% % % Music On Hold 93.3% % % Mute 70.0% % % Notes 44.8% % % Phone Cll Active 51.7% % % Retrieve 72.4% % % Ringer Select 43.3% % % Speed Dil 50.0% % % Spekerphone 63.3% % % Store 43.3% % % Switch Hook Control 27.6% % % Trnsfer 43.3% % % Volume 90.0% % % 6.09 Tble 1. Men Recognition nd Rtings Tests for the 23 Referents (N=30). 60% in ll 3 icon fmilies (Conference, Dilpd, Music On Hold, Spekerphone, Volume). As fmily, the bstrct icons hd the highest recognition mens for the 147

100 referent for Mute. Concrete icons were solely high for the referents Help Specific, nd Answer Ringing Cll. The Proposed icon groups hd the highest recognition mens for referents Hold, Messge, nd Notes. Both concrete nd proposed icon fmilies hd high men scores for referents Ringer Select, Speed Dil, nd Trnsfer. Overll men recognition percentges between the 3 icon groups were 5%0 to 59% with no significnt differences mong them (p<.05). Similrly, the icons with high men recognition percentges lso exhibited high men rting scores for the certinty tests. Rnk order correltions were lso t lest for these icons. Icons with low recognition mens hd low certinty levels nd semntic scores. For the SD tests, results showed tht correltions were high between the semntic scles nd the percent men of correct recognition regrdless of icon fmilies. Semntic Scles Abstrct Concrete Icons Proposed Icons Icons Concrete-Abstrct 0.707* 0.887* 0.888* Fmilir-Strnge 0.673* 0.871* 0.867* Meningful-Meningless 0.656* 0.886* Relted-Unrelted 0.668* 0.859* 0.714* Shrp-Dull 0.667* 0.878* 0.833* Simple-Complex 0.716* 0.879* 0.896* Tble 2. Correltions (rnk order) between Semntic Differentil Scores nd Percentge Correct Recognition (N=30); *p < A few insights cn be glened from the results. Culture cn be defined s something which lso influences the publicly vilble symbolic forms through which people experience, express nd understnd menings (Keesing, 1974). Of the 7 icons designed in Jpn, 5 hd high men percent of correct recognition. These results differed from those of Tudor (1994). Using Americn subjects the 7 icons designed in Jpn hd vrying levels of men vlues for correct recognition. Such difference my be reflection of Philippine socil condition where mjority of current computerised nd electronic pplinces in the Philippines re imported from Jpn. Americns on the other hnd re obviously more kin to be exposed to products they hve mde. Although USA still rnks s the mjor trding prtner of the Philippines followed by Jpn, the ltter rnks first in terms of the computer nd telecom products mrketed nd used loclly. Icons re lso commonly found in these products. These render the view tht culturl fmilirity nd mening of icons cn ffect their usbility when pplied s interfce tools in new or modern technologies. The SD results showed tht SD testing cn be vlid index of how messges of symbols like the icons re perceived nd understood. It is simple, inexpensive but 148

101 relible. Admittedly though, there re limittions in this pilot study. A distinct group of potentil users (educted nd computer literte) ws used s subjects. Future studies will need to utilise vrying groups of potentil users (students, professionls, experts, novices, etc.) cross different cultures of Asi. It will be quite useful nd interesting to study how icons in HCI s re recognised nd understood in these countries s compred to their counterprts in Europe nd Americ. References Keesing, R M (1974), Theories of Culture, Annul Review of Anthropology, vol 3, pp Kerlinger, F N (1973), Foundtions of Behviorl Reserch, Holt-Sunders Interntionl. Lin, R, nd Kreifeldt, J G (1992), Understnding the Imge Functions for Icon Design, Humn Fctors Society 36th Annul Meeting, Humn Fctors Society (St. Monic), pp Osgood, C E, Suci, G, nd Tnnenbum, P (1957), The Mesurement of Mening, University of Illinois Press. Tudor, L G. (1994), The Genertion nd Testing of Cndidte Telecommunictions Icons, AT & T Bell Reserch Ppers, pp Wood, W T, nd Wood, S K (1987), Icons in Everydy Life, Socil Ergonomics nd Stress Aspects of Work with Computers, pp

102

103 PAPER 2 EVALUATING PUBLIC GRAPHIC SYMBOLS USING MULTIPLE TEST PARAMETERS

104

105 EVALUATING PUBLIC GRAPHIC SYMBOLS USING MULTIPLE TEST PARAMETERS DOMINIC PAUL T. PIAMONTE 1, KJELL OHLSSON 2, nd JOHN ABEYSEKERA 1 1 Industril Ergonomics Division, Humn Work Sciences Deprtment, Luleå University of Technology, S Luleå, Sweden 2 Engineering Psychology Division, Humn Work Sciences Deprtment, Luleå University of Technology, S Luleå, Sweden Pul.Pimonte@rb.luth.se, Kjell.Ohlsson@rb.luth.se John.Abeyseker@rb.luth.se Phone: , Fx: Abstrct Grphic symbols require less spce, re visully more distinct, esy to lern nd cn help reduce the likelihood of errors when properly designed. With the current globliztion nd dvent of informtion technology, diverse user groups incresingly use them cross the world. However, generting nd testing grphic symbols in these groups cn be intricte. When user chooses which mong the present symbols will bring bout the trget function or informtion, severl fctors come into ply. Different tests my thus be helpful in determining which symbol or symbol sets re best. They lso need to be tested bsed on the contexts of their intended use. This study discusses the results of using multiple test methods in evluting videophone symbols mong groups of different Asin ntionlities. Correct ssocitions, confusions nd preferences re some of the importnt fctors to be considered when determining the grphic symbols to use. 1. INTRODUCTION Grphic symbols (icons nd pictogrms) hve become one of the strtegies to introduce technologies nd their products cross user groups of different cultures nd lnguges. Icons generlly refer to symbols tht re simple, concrete nd usully self-explntory of the ides, objects or functions they represent [30]. Pictogrms re usully more bstrct, conveying messges by nlogy or symbolism nd thus require certin lerning processes to be understood [3]. Grphic symbols hve become ttrctive s product interfces since they require less spce nd re non-text dependent. They cn even be visully more distinct thn texts mking them very useful in work groups of vrying cultures nd lnguges. In their pper on sfety pictogrms, Dvies, Hines, Norris, nd Wilson [5] mentioned tht in the light of the emerging Europen mrket, grphic symbols hve become very ttrctive since they hve the potentil to cross the lnguge brrier. Messges my need not lwys be trnslted or explined in the 13 lnguges of the Union. Regrding ergonomics of computerbsed work, grphic symbols cn fcilitte ccess to certin functions. Studies lso suggest tht when properly designed nd tested, they hve the potentil to help expedite work in emergency nd highly stressful settings such s rescue nd medicl (clinic nd hospitl) emergencies. The use of grphic symbols, however, is not devoid of problems. Foremost mong these is tht some symbols re simply not understood well [2]. The bove dvntges were cited s evident when the concepts being represented were well understood or concrete, nd not when ides become more bstrct. Snders [21] further dded tht the superiority of symbols could only be ttined if they contin smll mount of detil re sufficiently distinct in shpe nd re unmbiguous. Regrding older people, Dvis et l [5] lso cited the 1

106 studies of Esterby nd Hkiel [7] showing tht lthough symbols were esily recognizble, understnding their mening ws generlly poorer compred to younger people. 2. EVALUATING GRAPHIC SYMBOLS Numerous evlution studies hve been done in serch of wht, where nd how symbols cn best be used. A review of these studies revels t lest two criticl elements ffecting evlution results the methods used nd the subjects involved Current methodologies Vrious methods hve been utilized to evlute icons nd pictogrms. Some of the methods frequently used were ) ppropriteness tests, b) preference rtings tests, c) nming nd mtching tests d) comprehension tests, e) recognition nd recll tests, nd f) piredcomprison tests [4, 8, 9, 11, 12, 15, 17, 18, 19, 20, 24 nd 31]. Webb, Sorensen nd Lyons [28] described these nd other methods of evluting symbols s involving psychophysics, scling, recognition nd memory testing, s well s sttisticl modeling nd nlysis. Generlly, ppropriteness testing ws the preliminry procedure to screen severl cndidte designs. Mtching studies determined how well the symbols performed s set nd their likelihood of being confused with ech other [18]. However, owing to the complexity of how symbols re understood nd lerned, no single testing method is usully sufficient when determining which symbols to use especilly mong diverse user groups. More thn one mesure needs to be used, nd the relevnce of ech specific criterion depends on the purpose of the symbol nd where it will be used [6]. In method developed for the Europen Telecommunictions Stndrds Institute (ETSI), Böcker [3] utilized wht he clled the Multiple Index Approch (MIA) for empiricl evlution of grphic symbols. The method consisted of three-prt questionnires evluting seven indices deemed importnt when testing grphic symbols designed to pper simultneously s set (i.e. grphic symbols ppering on dilpds nd videophone terminls). He pointed out tht such pproch ws essentil in order to reflect typicl usge scenrios of these symbols. For exmple, testing cndidte videophone symbols should reflect the rel-life sitution wherein prospective user hd to choose which mong the simultneously visible symbols would bring bout specific function being sought (ex. switching on the cmer function of the videophone). The test simulted this by using cued response tests. It presented one symbol set nd one referent (function) t time nd subjects hd to select which symbol best represented the referent. The possible outcomes would then either be hit (choosing the right symbol), or miss (wrong symbol). Although the cued response test ws pproprite in this mnner, it would hve been more interesting if the spontneous identifiction or free recll test ws lso used. The free recll test is often regrded s the min stndrd of mesuring symbol comprehension nd often used to compre results using other test methods [6, 29]. The ETSI studies pointed out tht lthough the test on proper recognition of the right symbols to their referents (hits) ws the min index of performnce, other prmeters hd to be noted s well. The other non-hit possibilities would be confusion of symbols to one nother, filure to choose ny symbol t ll (no choice or missing vlues), subjective certinties nd representtiveness, s well s symbol nd symbol set preference [3]. 2

107 Lstly, testing grphic symbols needs proper contextuliztion s discussed by Wolff nd Woglter [29]. In rel-life, symbols do not pper independently of the environment where they re used. The contexts or situtions of how they re presented often contin hints or cues tht limit the possibilities of wht the symbols could be. Thus, there is need for dequte informtion bout the context on how the symbols will ctully pper [3, 29]. In Böcker s MIA, the introduction of the study nd description (illustrtion) of the bsic components nd functions of videophone prior to the tests somehow helped the subjects to contextulize wht the grphic symbols were ll bout Subject groups The other criticl fctor ffecting symbol evlution is the type of subjects involved in the studies. Prospective users involvement becomes more crucil when their bckgrounds re vstly different s in estern vs. western users, where one culture develops the products nd their symbols, nd nother culture/s use/s them. Keesing [10] defined culture s something tht lso influences the publicly vilble symbolic forms through which people experience nd express mening. Although the influence of culture on technology hs been recognized, empiricl studies re still indequte [1, 22, 25]. In his studies on culture, technology nd ergonomics, Shhnvz [23] reveled tht pictoril signs nd symbols for public use re very much relted to culture. He suggested tht the best wy to consider culturl spects in the design is to dpt user-centered design pproch. The users needs nd bckgrounds hve to be reflected in the design nd evlution of the cndidte symbols. However, icons nd pictogrms re generlly designed nd developed with western users in mind but trgeted for interntionl use. To compound these, the subjects used in some of the methods mentioned bove could hrdly even be described s representing typicl users (industril design students, professionls, nd even communiction employees) [26]. Obviously, ll these mtters need to be looked t nd studied more to determine which mong the symbols nd their sets re best or most pproprite. Considering tht most grphic symbols nd their devices re trgeted for worldwide use mong consumers of vrying cultures, the chllenge lies in developing test methods tht re inexpensive, yet esy to conduct nd comprehensive enough to cover the bove fctors. 3. OBJECTIVES OF THE STUDY In view of the bove, the present study ims to nlyze the merits of simple pencil nd pper tsks with multiple test prmeters in evluting grphic symbols mong subjects of diverse ntionlities. In the design of the methods, some criticl fctors bsed on the literture survey were considered. Specificlly, Böcker s MIA ws dopted but spontneous identifiction test ws dded for better comprison of the results. Since the grphic symbols studied were designed nd tested in the West (Europe), subjects from the Est (Southest Asi) were used to help shed some light if culture indeed cn influence symbol comprehension. 3

108 4. METHODS AND MATERIALS 4.1. Symbols used in the study In the form of booklets, severl grphic symbols representing videophone referents or functions were tested. Ech symbol ws pproximtely 1.2 x 1.2 cm in size. The study tested three sets of seven videophone symbols representing seven videophone functions or referents for totl of twenty-one symbols (Figure 1). All the symbols were bsed on the ETSI study with one of these sets (Set 1) recommended by ETSI s the stndrd. The seven-videophone functions tested were the following [3]: ) Videophone / Telephone: upgrding / downgrding the cll b) Cmer on / off: turning on nd off picture trnsmission c) Microphone on / off: turning on nd off sound trnsmission d) Selfview on / off: turning on nd off selfview function e) Document Cmer on / off: switching between document nd person cmer f) Still Picture on / off: turning on nd off screen freeze g) Hndsfree on / off: switching between hndset nd hndsfree modes 4.2. Subjects One hundred ninety-two (99 mles nd 93 femles) university students nd employees of smll nd lrge compnies from four Southest Asin countries cted s subjects in the study. There were 48 subjects ech from Indonesi, Mlysi, Philippines nd Thilnd. The men ge ws 25.4 yers (SD = 6.4), rnging from 17 to 52 yers old. All were computerliterte or hd t lest experienced using computers or computer-relted products for the pst 2-5 yers Procedure The tests were in the form of questionnires. They were done in smll groups in ech country, lsting for bout minutes. Although ll the subjects could red nd spek English, trnsltions of the questionnires were lso provided in ech country s ntive lnguge. Subjects were rndomly given one of three versions of the questionnires. They were shown n illustrtion of videophone. Its min prts nd functions were then crefully discussed without reveling the ctul referents to be tested. For exmple, terms such s selfview nd document cmer were not stted directly. Rther they were told tht by pushing certin keys, one could see oneself or view documents. Afterwrds, instructions were given on how to go bout the different test prts. Questions were entertined prior to the dministrtion of the tests. Emphsis ws given on voiding omissions in trils in order to get bck to them lter on. The order of the tests would lso be strictly followed; tht is, Prt One followed by Prts Two nd Three. 4

109 In the spontneous identifiction or free recll test (Prt One), ech pge contined one set of the seven videophone symbols, for totl of three pges for the three symbol sets. On spces provided for, they would write wht videophone function they thought ws represented by ech symbol. They were lso sked to rte the level of certinty for ech of their nswers using seven-point rting scle (from very certin to very uncertin ). In the cued response test (Prt Two), the subjects first red referent nd its description. Then they hd to choose one symbol from set of seven symbols which they thought best represented the referent in question. Ech pge contined one referentdescription nd one set of symbols. They were lso sked to rte their certinties for their nswers using the 7-point rting scles. There were seven videophone referents tested on three sets of symbols rendering totl of twenty-one trils in Prt Two. For the lst prt, three symbols (one for ech set) together with the referent they represented were shown. The subjects hd to choose one symbol they thought best represent tht referent. Next, with the three symbols sets presented together, they hd to choose the set they preferred most. In ll, the subjects would choose seven symbols to represent the seven referents nd one symbol set they preferred most Experimentl Design The study utilized 2 x 5 x 3 (Occuption x Country x Symbol Set) repeted mesures design with 2 between nd 2 within fctors. Between fctors were those of occuption (students nd employees) nd country (Indonesi, Mlysi, Philippines, Thilnd nd Sri Lnk). The within fctor consisted of the symbol sets (Sets 1, 2 nd 3) nd test types (spontneous identifiction nd cued response tests). The results for the subject groups by occuption in ech country were collpsed since the focus in this prt of the study ws response differences between countries for different test prmeters. In ll three prts of the tests, the order of the symbol sets ws counterblnced cross ll subject groups. Three orderings of the symbol presenttion nd referents were generted nd rndomly distributed in ech subject group to minimize order nd lerning effects. The test prmeters studied were the following: ) Hit rtes (men percentges of correct identifiction/recognition in Prts One nd Two) b) Subjective certinty rtings using 7-point scles for nswers in Prts One nd Two c) Missing vlues (No nswers in Prts One nd Two) d) Confusions for ech videophone symbol (totl numbers nd frequency distributions) e) Symbol nd symbol set preferences (percent distribution, Prt Four) 5. RESULTS 5.1. Hit rtes nd subjective certinties In the spontneous identifiction or free recll test, hit (correct nswer) occurs when the subject correctly identifies the symbol by writing its referent or function. For the cued 5

110 response test, hit pertins to correctly choosing the symbol representing the referent being sked. A one-between (country) nd two-within (symbol set nd test type) repeted mesure ANOVAs were performed to compre the results in the two tests. Post hoc tests nd pirwise multiple comprisons using Tukey s HSD were done for the significnt min effects (symbol set, test type, nd country t p < 0.05). In testing symbols intended for public use, the Orgniztion for Interntionl Stndrdiztion s ISO 9186 recommended t lest 67% comprehension of the symbols [5]. This stndrd ws intended more for symbols or signs ppering individully rther thn s group. Due to lck of more suitble guideline, ISO 9186 ws nonetheless dpted to roughly nlyze nd compre the results of the identifiction nd recognition tests. Figures 2-h contin the hit rtes in the spontneous identifiction (free recll) nd the cued-response tests respectively for the three symbol sets.. Between symbol sets. Overll, Set 1 s symbols hd the highest hit rtes compred to Sets 2 nd 3 in lmost ll of the 7 referents or functions tested. The ANOVAs showed Set 1 symbols with significntly higher hit rtes thn Set 2 symbols in ll seven referents, (F(2, 376), p < 0.05). They were lso significntly higher thn Set 3 symbols except for microphone, nd selfview. The hit rtes of Sets 2 nd 3 were not significntly different from ech other except, for microphone, F(2, 376) = , p < 0.001, selfview, F(2, 376) = 67.61, p < 0.001, nd videophone, (F(2, 376) = , p < 0.001, where Set 3 s hit rtes were higher. Although Set 1 hd the highest hit rtes, only 3 of its symbols hd hits bove the 67% criteri of ISO 9186 ( cmer 73.4%, microphone 83.8% nd videophone 78.4%). Set 3 hd 2 of its symbols bove 67% ( microphone 81.0% nd videophone 70.0%). All of Set 2 s symbols hd hits below 67%. b. Between tests. Prt One ws much hrder tsk compred to Prt Two s proven in the results. Tking between tests s min effect, the hit rtes in the cued response tests (Prt Two) were significntly higher thn in the spontneous identifiction tests (Prt One) for ll the 7 referents. However, when considering ISO 9186 in Prt One, only the symbol of Sets 1 nd 3 for microphone met the 67% correct criterion. In Prt Two, the hit rtes of Sets 1 nd 3 for cmer, microphone nd videophone nd Sets 1 nd 2 s symbols for document cmer hd hits beyond 67%. c. Between countries. Significnt differences on hit rtes between countries were noted only for the hndsfree (F(3, 188) = 3.40, p < 0.019), nd videophone (F(3, 188) = 2.92, p < 0.035). In the hndsfree symbols, subjects from Thilnd hd significntly higher hit rtes thn the three other countries. The videophone symbols showed Mlysi hving lower hit rte compred to Indonesi nd Thilnd. However, if the 67% criterion ws considered, only Thilnd hd hit rte bove this level ( cmer 69.4%). d. Subjective certinties. Subjective certinty rtings for both Prts One nd Two re lso shown on Figures 2-h (line grphs). The rtings were bsiclly consistent with the identifiction nd cued response dt. Symbols with high hit rtes exhibited high certinty rtings s well - gin, mostly symbols from Sets 1nd 3. Spermn s rnkorder correltion ws lso pplied between the prmeters hit rtes, subjective certinties nd symbol preferences (Tble 1). Correltion between hits nd certinties is highest for Set 1 s cmer, hndsfree, microphone nd videophone. There 6

111 were more significnt correltion between hit rtes nd subjective certinties thn hit rtes nd preferences Confusion mtrices Confusions were instnces when symbol ws ssocited to the wrong referents. They re nother wy of nlyzing incorrect nswers or misses. In this study, focus would be on those derived during the cued response test (Prt Two). Bsed on the confusion mtrices on Tbles 2 nd s summrized on Tble 3, Set 2 hd the highest instnces of confusions nd Set 1 hving hd the lest confusions cross ll countries. This ws expected since the set (Set 2) with the most misses would normlly hve the most confusions. Confusion instnces re criticl especilly when the symbols re presented s set or simultneously. They indicte which mong the symbols pper similr nd thus would merit re-design. Looking t ll the three sets, the symbols for "videophone" nd "selfview" hd the highest instnces of being confused with other functions. The symbols with the lest instnces of being confused with other referents were "hndsfree" (Sets 1 nd 3), "cmer" (Set 2), nd "document cmer" (Set 3). The different countries were quite similr in the confusions of the symbols esily identifying fmilir symbols ("cmer" nd "document cmer") nd confusing them with other functions. On the other hnd, the symbols for "selfview" nd "videophone" were commonly confused with other referents since they were too generl to specify 5.3. Missing vlues nd preferences Missing vlues refer to situtions when subjects did not write ny nswer in Prts One nd Two (Tble 4). Prt One, being the hrder tsk, hd more missing vlues thn Prt Two. Purely bstrct symbols lso tend to be identified lest s shown by the results in Set 2 hving the highest missing vlues for both Prts. Specificlly, the symbols for "selfview", nd "microphone" of Set 2 hd the most number of missing vlues owing to their bstrctness. There were no mjor differences between countries for missing vlues in Prt One. Though in Prt Two, Indonesi hd missing vlues bit higher compred to Mlysi, the Philippines nd Thilnd. Tbles 5 nd 6 contin the men symbol nd symbol set preferences of ll countries. The symbols of Set 1 ws the preferred symbol by t lest 50% (significnt t p<0.05) of the subjects for document cmer, hndsfree, still picture, nd videophone. For the other symbols, the subjects preferred Set 2 s cmer nd Set 3 s microphone. Between sets, bout sixty-five percent of the subjects preferred Set Combined results By considering the results of both the hit rtes nd subjective certinties in Prts One nd Two, s well s confusions nd preferences, Set 1 s symbols performed best in five of the seven referents tested. Exceptions were in symbols for microphone, nd selfview, where symbols of Sets 1 nd 3 performed eqully better. If the results of the confusion mtrices nd preferences were dded, Set 1 s selfview nd Set 3 s microphone would be the best symbols for these referents. Set 2 s symbols were mostly bstrct nd mbiguous, grnering the lowest scores in most of the test prmeters used in the study. 7

112 It is importnt to note lso tht in Prt One ll countries performed poorly, registering hit rtes of t lest 67% in only up to 3 of the 7 referents, mostly with Set 1 s symbols (Figures 2-h). These improved to 4-6 symbols with hit rtes of t lest 67% in Prt Two. Between countries, Thilnd hd significntly higher hits for hndsfree, microphone (with the Philippines), still picture (with Mlysi), nd videophone (with Indonesi). Although Mlysi nd the Philippines hd hits of 67% or more in only 1-2 of the symbols in Prt One, they registered the biggest improvements in Prt Two, hving 5-6 symbols with t lest 67% hit rtes (Sets 1 nd 3). 6. DISCUSSION 6.1. Hit rtes nd subjective certinties Grphic symbols hve become populr prt of interfces of modern devices. They help mke the products pper more ttrctive nd useful with miniml spce demnds. But to the uninitited user, grphic symbols cn lso esily mke him confused, wsting unnecessry time nd effort trying to mke sense of everything. When user chooses which mong the present symbols will bring bout the trget function or informtion, severl fctors come into ply. Different tests my thus be helpful in determining which symbol or symbol sets re best. This study exmined some of these fctors ffecting symbol understnding using simple nd inexpensive tests. The most common guge of symbol performnce is the hit rte. The hit rte is the correct ssocition between referent nd symbol (videophone symbols in this cse). In this study, hit rtes were generted using two types of tests the open-ended or spontneous identifiction test nd the cued response test. The former is usully designed to mesure initil ( priori) mening while the ltter tests the ssocitiveness of symbol to its intended mening or referent. Prts One nd Two simulte (though in limited mnner) rel videophone cll scenrios wherein the symbols re plced on the terminl with one function to be performed t time. Prt One is sitution kin to using the technology the first time with miniml knowledge nd instructions. Prt One s results showed tht grphic symbols could be quite difficult to understnd or comprehend for the Asin subjects bsed on the very low hit rtes. None of the subjects hd ever used videophone nd this finding demonstrted tht new users could indeed esily experience difficulties in using new technologies with grphic symbols s the sole interfce nd no other cues (lerning ids, help functions) redily present. The higher hit rtes of Prt Two (cued responses) vlidted this point. With proper nd simple verbl nd written instructions, improvements in symbol recognition could usully be chieved [14, 22]. Performing Prts One nd Two re essentil in determining which symbols to retin, re-design or discrd. Improving performnce from Prt One to Prt Two shows tht the symbols cn be cceptble, but proper cues nd lerning ids re very importnt to retin recognition nd understnding. Very low hit rtes in both tests (such s those in Set 2) denote tht the symbols re poorly understood nd re not suitble for use by diverse groups. Subjective certinties indicte the subjects level of sureness tht their choices of symbols to prticulr referents re right. High certinty levels of Set 1 s microphone nd videophone ptly showed these ptterns. Furthermore, the hit rtes generlly hd significnt positive correltion with their subjective certinties (Figures 2-h). Since there were more significnt correltion s between hit rtes nd subjective certinties thn hit rtes 8

113 nd preferences, these my suggest preference s more reflection of esthetics nd personl choice nd my not lwys reflect correct understnding of the symbols Symbol distinctiveness the confusion fctor As mentioned erlier, using hit rtes lone cnnot revel the dynmics between symbols, especilly when they re designed to pper together s set. Confusion between symbols or instnces when symbol is ssocited to the wrong referents do occur nd re thus lso importnt. When symbols for the sme referent hve similr hit rtes confusion mtrices re helpful to determine which is better nd more distinct. In the cse of the symbols for selfview, lthough Sets 1 nd 3 pper to hve similr hit rtes, the former seemed to be better by hving much lesser instnces of confusion. The sme ws true for the videophone symbols of Sets 1 nd 3. They hd similr hit rtes but the videophone of Set 1 seemed to be more distinct. Confusing one symbol to nother my even be more hzrdous thn miss since the former represents how often symbol is mistkenly chosen under different referent conditions thereby decresing its usbility [3]. When confusions re concentrted to one or few referents, they lso represent the degree of confusing symbol to nother symbol presented t the sme time. For exmple, Set 1 s microphone ws mostly ssocited with the referent hndsfree by Indonesin nd Mlysin subjects. Such situtions could men tht the symbols might be so similr to ech other tht re-design is in order to mke them more distinct from one nother Missing vlues nd preferences Missing vlues is nother importnt wy to guge the qulity of symbol [3]. They re situtions wherein the subjects did not know the nswer or no ide of symbol s mening. In ctul videophone usge scenrio, the user could not identify which symbol represented the desired ction. This fctor, lthough seldom used, is still very helpful in identifying symbols subjects find difficult to understnd nd re esily derived from tests such s those in Prts One nd Two. Preferences represent the subjects personl judgement which of the symbols nd sets best represent the referents in question (Tbles 5 nd 6). Tble 6 shows clerly tht Set 1 ws the preferred set with five of its seven symbols significntly preferred over their counterprts. Since preference represent esthetics (symbol s ppel to subject) it cn be very helpful when deciding symbols which hve very similr results in ll the other performnce mesures The culturl fctor One s ntionlity is common indictor of one s culture. This study ttempted to study if subjects from Southest Asin countries of diverse cultures (ntionlity) differ in their recognition nd understnding of grphic symbols. The results bove showed tht differences s well s similrities occurred between the Asin subject groups studied. For the ltter, ll of them performed very poorly in Prt One, hving high hits only for microphone nd cmer (Set 1 nd 3). This could be explined by the fct tht these symbols re very fmilir nd concrete representtions of very fmilir functions. The high improvements in hit rtes in Prt Two likewise showed tht with proper instructions nd cues, Asin subjects would not hve mjor problems recognizing grphic symbols. Differences ly more when the symbols were probbly new, unfmilir or bstrct such s the symbols for hndsfree, wherein Thilnd hd higher hits compred to the other countries, nd videophone with 9

114 Indonesi nd Thilnd performing better thn Mlysi. The missing vlue fctor lso helped qulify the misses or wrong nswers. Among the 4 countries, Indonesi hd the highest number of no nswers with Thilnd nd the Philippines hving the lest. But compred to the Europen study, the Asin subjects hd lesser instnces of no nswer [3]. This fctor is interesting nd requires further study since missing vlues could represent not only the inbility to recognize the symbols ( did not know, no ide ) but lso the level of conservtism when nswering tests. 7. CONCLUSIONS In this study, ETSI s recommended symbols (Set 1) turned to be the best nd most preferred by the Asin subjects but my be poorly understood if they re introduced without dequte instructions nd prctice. Multiple test prmeters were very vluble in determining initilly which mong the symbols will be redily understood nd which will require more lerning or even re-design. They need not be expensive nd could be done economiclly nd conveniently using simple pper nd pencil tsks. Among the test prmeters, the hit rte fctor is still very importnt but insufficient, nd by itself is too generl n index. Certinty levels nd missing vlues (no nswers) could help further qulify the levels of hit rtes ttined by the symbols. The extent symbol is wrongly ssocited to other referents (confusion) is likewise importnt, especilly if together with other symbols they re designed to pper simultneously s set. And when symbols re intended for interntionl use, using multiple test prmeters become more useful in identifying not only the best symbols to use but lso the chnges or improvements needed by the different user groups. 8. ACKNOWLDGEMENTS Acknowledgements re sincerely extended to the following: to Mr. Mrtin Böcker for grciously shring his thoughts nd opinions nd ll possible ssistnce regrding their ETSI studies on icons nd pictogrms; nd to the Swedish Telecommunictions Reserch Bord (KFB) for the finncil nd logisticl supports; to the ASEAN Ergonomics Society (Indonesi, Mlysi, Philippines nd Thilnd) for helping fcilitte the conduction of the tests in these countries. 9. REFERENCES [1] J. D. A. Abeyseker, H. Shhnvz nd L. J. Chpmn, Ergonomics in developing countries, Advnces in Industril Ergonomics nd Sfety II, Tylor nd Frncis, London, 1990, pp [2] J. W. Brelsford, M. S. Woglter nd J. A. Scoggins, Enhncing comprehension nd retention of sfety-relted pictorils, Proceedings of the Humn Fctors Ergonomics Society 38 th Annul Meeting, 1994, pp

115 [3] M. Böcker, A multiple index pproch for the evlution of pictogrms, Proceedings of the 14th Interntionl Symposium on Humn Fctors in Telecommunictions, Drmstdt, Germny, R. v. Decker's Verlg, G. Schenck GmbH, Heidelberg, 1993, pp [4] A. M. Clrke nd S. Brown, Pictogrms in multipoint videotelephony: n interntionl study, Contemporry Ergonomics 1992, Lovesey, E. J. (ed.), Tylor nd Frncis, London, 1992, pp [5] S. Dvies, H. Hines, B. Norris nd J. Wilson, Sfety pictogrms: re they getting the messge cross? Applied Ergonomics, 29 (1998), [6] R. Dewr, Design nd evlution of grphic symbols, Proceedings of Public Grphics, Utrecht, Netherlnds, University of Utrecht, Deprtment of Psychonomics, 1994, pp [7] R. S. Esterby nd S. R. Hkiel, Field testing of consumer sfety signs: the comprehension of pictorilly presented messges, Applied Ergonomics, 12 (1981), [8] P. Green nd R. W. Pew, Evluting pictogrphic symbols: n utomotive ppliction, Humn Fctors 20 (1978), [9] S. Jones, Stereotipy in pictogrms of bstrct concepts, Ergonomics, 26 (1983), [10] R. M. Keesing, Theories of culture, Annul Review of Anthropology, 3 (1974), [11] R. Lin nd J. G. Kreifeldt, Understnding the imge functions for icon design, Innovtions for Interctions. Proceedings of the Humn Fctors Society 36th Annul Meeting, Atlnt, Georgi, Humn Fctors Society, Snt Monic, 1992, pp [12] R. Lin, J. G. Kreifeldt nd C. F. Chi, A study of evlution design sufficiency for iconic interfce design: the design perspective, Contemporry Ergonomics 1992, Lovesey, E. J. (ed.), Tylor nd Frncis, London, 1992, pp [13] K. N. Lodding, Iconic interfcing, IEEE Computer Grphics nd Applictions, 3 (1982), [14] A. M. L. Lund, Expert rtings of usbility mxims, Ergonomics in Design, 5 (1997), [15] J. Mckett-Stout nd R. Dewr, Evlution of symbolic public informtion signs, In Proceedings of Humn Fctors Society Conference, 1981, pp [16] M. C. Mguire, A review of humn fctors guidelines nd techniques for the design of grphic humn-computer interfces, Computers & Grphics 9 (1985), [17] R. L. Mgyr, Assessing icon ppropriteness nd icon discriminbility with pired comprison testing procedure, In Proceedings of the Humn Fctors Society 34th Annul Meeting, 1990, pp

116 [18] P. R. Noln, Designing screen icons: rnking nd mtching studies, Proceedings of the Humn Fctors Society 33rd Annul Meeting, Humn Fctors, Snt Monic, 1989, pp [19] Y. Rogers, Evluting the meningfulness of icon sets to represent commnd opertions, Proceedings of Humn Computer Interction '86, British Computer Society Ed., 1986, pp [20] A. Slsoo, Towrds usble icon sets: cse study from telecommunictions engineering, In: Countdown to the 21st Century. Proceedings of the Humn Fctors Society 34th Annul Meeting, Orlndo, Florid, October 8-12, Vol. 1, Snt Monic: Humn Fctors Society, 1990, pp [21] A. F. Snders, Informtion processing nd design, Chpter 29: Ergonomics, Work System Design, Encyclopedi of Occuptionl Helth nd Sfety, ILO, Genev, 1998, pp [22] H. Shhnvz, Trnsfer of Technology to Industrilly Developing Countries nd Humn Fctors Considertions, Center for Ergonomics of Developing Countries, Luleå: Luleå University Press, Sweden, [23] H. Shhnvz, Culturl differences, Chpter 29: Ergonomics, Designing for Everyone, Encyclopedi of Occuptionl Helth nd Sfety, ILO, Genev, 1998, pp [24] R. B. Stmmers, Judged ppropriteness of icons s predictor of identifiction performnce, Contemporry Ergonomics 1990, Lovesey, E. J. (ed.), Tylor nd Frncis, London, 1990, pp [25] P. Sukviriy nd L. Morn, User interfce for Asi, Designing Interfces For Interntionl Use, Nielsen, J. (ed.), Elsevier, Amsterdm, 1990, pp [26] L. Tudor, The genertion nd testing of cndidte telecommunictions icons, Interntionl Conference in Humn-Computer Interction Proceedings, St. Petersburg, Russi, August 2-6, 1994, pp [27] P. Vor, M. Helnder, H. Swede nd J. Wilson, Developing guidelines for symbol design comprison of methodologies, In Proceedings of Interfce, 1991, pp [28] J. M. Webb, P. F. Sorenson nd N. P. Lyons, An empiricl pproch to the evlution of icons, SIGCHI Bulletin, 21(1989), [29] J. S. Wolff nd M. S. Woglter, Comprehension of pictoril symbols: effects of contexts nd test method, Humn Fctors, 40 (1998), [30] W. T. Wood nd S. K. Wood, Icons in Everydy Life, Socil Ergonomics nd Stress Aspects of Work with Computers, 1987, [31] H. L. G. Zwg, Predicting the public's comprehension of grphic symbols, In Contemporry Ergonomics 1989, E. D. Megw, ed., Tylor nd Frncis, London, 1989, pp

117 TABLE 1. Correltion coefficients (Spermn s rho) of hit rtes with subjective certinties nd preferences. Symbol Set Subjective Certinties Preferences * 0.162* Cmer * * Document * Cmer * * Hndsfree * * Microphone * Selfview * * * Still Picture * * 0.190* * Videophone * *correltion with hit rte is significnt t P<

118 Tble 2. Confusion mtrices per country (n = 48 per country) with hits (highlighted boxes), misses (columns) nd confusions (rows) for the three symbol sets. First column in ech set contins the seven videophone symbols nd the second rows the seven referents or functions. INDONESIA Set 1 Set 2 Set 3 HF SV SP C VP D M H SV SP C VP D M HF SV SP C VP D M Hndsfree Selfview Still Picture Cmer Videophone Doc. Cmer Microphone MALAYSIA HF SV SP C VP D M H SV SP C VP D M HF SV SP C VP D M Hndsfree Selfview Still Picture Cmer Videophone Doc. Cmer Microphone PHILIPPINES HF SV SP C VP D M H SV SP C VP D M HF SV SP C VP D M Hndsfree Selfview Still Picture Cmer Videophone Doc. Cmer Microphone THAILAND HF SV SP C VP D M H SV SP C VP D M HF SV SP C VP D M Hndsfree Selfview Still Picture Cmer Videophone Doc. Cmer Microphone Note: HF =Hndsfree, SV = Selfview, SP = Still Picture; CA = Cmer; VP = Videophone; DC = Document Cmer; MP = Microphone. 14

119 Tble 3. Totl number of confusions per country in Prt Two (Cued response tests). Set 1 Set 2 Set 3 Ind. Ml. Phi. Th. Ind. Ml. Phi. Th. Ind. Ml. Phi. Th. Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Selfview Tble 4. Missing Vlues per country (Note: Ind.=Indonesi, Ml.=Mlysi, Phi.=Philippines, Th.=Thilnd. The first column in ech country denotes missing vlues in Prt One nd the second column for missing vlues in Prt Two.) SET 1 SET 2 SET 3 IND MAL PHI THA IND MAL PHI THA IND MAL PHI THA Cmer Doc. Cmer Hndsfree Microphone Still Picture Videophone Totl

120 TABLE 5. Symbol preferences in (%) TABLE 6. Symbol set preferences with ll countries combined (N = 240). in (%) per country. Symbol Preferences Set Preferences Set 1 Set 2 Set 3 Set 1 Set 2 Set 3 Cmer Indonesi Doc. Cmer Mlysi Hndsfree Philippines Microphone Thilnd Selfview Mens Still Picture Videophone Referents Set 1 Set 2 Set 3 Cmer Document Cmer Hndsfree Microphone Selfview Still Picture Videophone Figure 1. The seven referents nd the three sets of videophone symbols used in the study. 16

121 Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert.-3 Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert.-3 Hits (%) MP VP CA DC HF SP SV Referents Figure 2. Prt One, Indonesi Certinties Hits (%) DC VP MP CA SV SP HF Referents Figure 2b. Indonesi, Indonesi Certinties Hits (%) Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert MP CA VP DC SP HF SV Referents Figure 2c. Prt One, Mlysi Certinties Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert.-3 Hits (%) Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert VP DC MP SP SV CA HF Referents Figure 2d. Prt Two, Mlysi Certinties Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert.-3 Hits (%) MP CA VP DC HF SP SV Referents Certinties Hits (%) VP MP CA DC SV SP HF Referents Certinties Figure 2e. Prt One, Philippines Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert.-3 Figure 2f. Prt Two, Philippines Hits 1 Hits 2 Hits 3 Cert.-1 Cert.-2 Cert.-3 Hits (%) MP VP CA SP HF SV DC Referents Figure 2g. Prt One, Thilnd Certinties Hits (%) VP MP DC CA SV HF SP Referents Figure 2h. Prt Two, Thilnd Certinties Figures 2-h. Prts One nd Two Hit rtes nd Subjective Certinties for the 3 symbol sets. Br grphs represent the Hit rtes (%) while line grphs represent Subjective Certinties (n = 240. CA=Cmer, DC=Document Cmer, HF=Hndsfree, MP=Microphone, SV=Selfview, SP=Still Picture, VP=Videophone). 17

122

123 PAPER 3 THE RELEVANCE OF ERROR ANALYSIS IN GRAPHICAL SYMBOLS EVALUATION

124

125 THE RELEVANCE OF ERROR ANALYSIS IN GRAPHICAL SYMBOLS EVALUATION Dominic Pul T. Pimonte Industril Ergonomics Division Humn Work Sciences Deprtment Luleå University of Technology, Luleå, Sweden ABSTRACT In n incresing number of modern tools nd devices, smll grphicl symbols pper simultneously in sets s prts of the humn-mchine interfces. The presence of ech symbol cn influence the other s recognisbility nd correct ssocition to its intended referents. Thus, side from correct ssocitions, it is eqully importnt to perform certin error nlysis of the wrong nswers, misses nd confusions, nd even lck of nswers. This reserch imed to show how such error nlyses could be vluble in evluting grphicl symbols especilly cross potentilly different user groups. The study tested three sets of icons representing seven videophone functions. The methods involved prmeters such s hits, confusions, missing vlues nd misses. The ssocition tests showed similr hit rtes of most symbols cross mjority of the subject groups. However, exploring the error ptterns helped detect differences in the grphicl symbols performnces between subject groups, which otherwise seemed to hve similr levels of recognition. These re very vluble not only in determining the symbols to be retined, replced or re-designed, but lso in formulting instructions nd other ids in lerning to use new products fster nd more stisfctorily Advent of Grphicl Symbols 1. INTRODUCTION Under the current globlistion, it is n incresing chllenge for ergonomists to ensure the usbility, comfort nd sfety of new technologies nd their products cross vried user groups. Grphicl symbols (icons nd pictogrms) ply n importnt prt in chieving this gol. From their trditionl use of ppering singly or in isoltion to denote loctions or services, they hve strted to pper simultneously in sets, s ctul prts of the humnmchine interfces of modern devices. They re now lso used to present the concepts, ides or objects, nd functions of the technology nd products where they re used. When properly designed, tested nd tught, grphicl symbols cn help introduce new technologies nd products to different user groups cross the world regrdless of culture or lnguge groups Methods in Evluting Grphicl Symbols As in ll products designed for public use, grphicl symbols need to be evluted which mong them re best suited cross ll intended user groups. The test methods to be employed re often dependent on the kind of study being pursued. Different methods hve been used in evluting symbols such s icons nd pictogrms (Mgyr, 1990; Noln, 1989; Webb, Sorenson & Lyons, 1989, nd Vor, Helnder, Swede & Wilson, 1991). If symbols re intended s prts of interfces of devices for interntionl use or for stndrdistion, different tests re usully needed. For exmple, the Interntionl Stndrds Orgnistion (ISO) hs come out with the ISO It is six-stge procedure for the development nd testing of 513

126 public informtion symbols (Zwg, 1989). Its mjor portions re the comprehensibility judgement tests, comprehension test with suggested comprehension level of t lest 67%, nd mtching tests. The Americn Ntionl Stndrds Institute likewise hs the ANSI Z535.3 (ANSI, 1991) describing the methods of evluting the comprehensibility of grphicl symbols. It utilises four ctegories nmely correct nswer, wrong nswer, criticl confusion nd no nswer. It further recommends comprehension level of t lest 85%. On the other hnd, the Interntionl Telecommunictions Union (ITU-T) hs F.910. Recommendtion F.910 endorses symbol testing procedure composed of four prts. It involves the determintion of need for new symbols, the cretion nd evlution of the new designs, nd the selection nd pprovl (ITU-T, 1995). Another importnt spect of symbols evlution is the need to explin nd test the functions (represented by the symbols) in mnners reflecting typicl user scenrios (Tudor, 1994). As mentioned erlier, numerous modern computer-bsed devices contin smll grphicl symbols simultneously ppering s groups s prt of the generl control interfce. The most pproprite wy to evlute the symbols in such cses is to present ech cndidte set of symbols ginst one trget referent t time. For exmple, in typicl user scenrio prospective user intends to use device or equipment nd is confronted with controls represented by set of grphicl symbols. He/she then hs to choose which mong these symbols correspond to his desired function. One such method ws developed by Böcker (1993) with the Humn Fctors Technicl Committee of the Europen Telecommunictions Stndrds Institute (ETSI). They evluted videotelephony symbols using multiple index pproch (MIA) which cn lso be used in testing symbols for other commercil products. The MIA tested severl sets of cndidte videophone symbols by presenting ech set to the subject with only one referent t time. This pproch enbled the tester to collect seven indices tht could help in the finl selection of symbols. The indices were hit rtes, flse lrm rtes, missing vlues, subjective certinty nd suitbility, symbol nd symbol set preferences (Böcker, 1993) Importnce of Anlysing Errors The hit rte in MIA reflected the levels of correct ssocitions of the symbols to their intended referents, nd ws the min index of performnce. However, it hs been emphsised tht hit rtes lone cnnot be the sole bsis of determining how well symbol is understood especilly if it ppers together with other symbols. Anlysing the ptterns or chrcteristics of errors such s misses, flse lrms or confusions with other symbols, nd even missing vlues re eqully importnt (Böcker, 1993 nd ETSI, 1993). Misses re instnces of selecting the wrong symbol in the context of prticulr function being sought. Their ptterns of distribution cn give some knowledge if the referents themselves re conceptully cler to the users. Flse lrms re instnces of symbol being wrongly selected under different non-corresponding functions or referents. Missing vlues re instnces wherein no response or no nswers were given. A hit or correct ssocition is just one of the possible outcomes in symbol testing. Misses nd other forms of errors re the others tht need to be closely studied s well. These considertions become more vluble when fced with potentilly different user groups. This pper imed to show how such nlyses of errors, i.e. their distribution pttern nd other chrcteristics, could be vluble in evluting grphicl symbols especilly cross potentilly different user groups. The study ws prt of n interntionl project in evluting telecommuniction icons using multiple inter-relted test prmeters. Some of these re the errors or so-clled non-hit prmeters, which re the foci of this pper. As stted erlier, 514

127 grphicl symbols lwys hve the potentil of widespred usbility mong different user groups, but they still need to be tested when the groups in question re s diverse s in Asi. This pper focused on some countries in Southest Asi - region chrcterised by n immense diversity in lnguge nd culture. Modern products nd the symbols they contin re often designed nd tested in the west but trgeted for worldwide use. Thus, region such s the Southest presents big chllenge to the designers of such products to mke them usble nd thus, commercilly vible s well. It is then essentil to evlute products nd their interfces with these potentil users in mind. It is hypothesised tht simple recognition tests eliciting hit rtes my fil to exhibit differences on how symbols re understood by different user groups. Other prmeters, s discussed bove, would be utilised to detect other possible differences in symbol recognition by different user groups. In the design of the study, the MIA by Böcker (1993) s well s Tudor s (1994) comments nd recommendtions were gretly considered. 2. METHODS 2.1. Subjects Two hundred forty university students nd employees nd professionls (127 mles nd 113 femles) from smll nd lrge compnies from five Asin countries prticipted in this study. These countries were Indonesi, Mlysi, Philippines, Thilnd nd Sri Lnk. There were forty-eight subjects per country with equl numbers of students nd employees, ll of whom were computer-literte or hd t lest experienced using computers or computer-relted products for the pst 2-5 yers. At the time of the study, none of the subjects ever experienced using videophone. The men ge ws 25.1 yers (SD = 6.1), rnging from yers old. These prticipnts cme from the mjor cities of these countries nd the tests conducted in their schools, offices nd homes Mterils The stimuli were three sets of symbols representing seven referents or functions of videophone for totl of twenty-one symbols (Figure 1). The referents were cmer, document cmer, hndsfree, microphone, selfview, still picture nd videophone. Both referents nd symbols were bsed on the ETSI study heded by Böcker (1993). The sets were grouped bsed on the outcome of the ETSI tests. Set 1 hd the best performnce bsed on the MIA tests (nd ws recommended by ETSI s the stndrd), followed by Set 3, nd Set 2 s lst. These three sets were chosen lso bsed on their symbols generl ttributes such concreteness or bstrction, nother fctor being studied by the project. The questionnires were divided into three prts: symbol identifiction/subjective certinty tests (Prt I), symbol ssocition/subjective certinty tests (Prt II), nd symbol nd set preferences (Prts III nd IV). Three versions of the questionnires with different orders of the symbols nd sets were generted nd used in the study. This pper would del on the results of Prt II Procedure The tests were done in smll groups in ech country in their schools, homes or offices, lsting for bout minutes. The subjects were rndomly given one of the three versions of the 515

128 questionnires. The versions differed only in symbol nd referent orderings. An illustrtion of videophone ws shown nd its generl functions were then discussed. Afterwrds, instructions were given on how to go bout the different test prts. Questions were entertined prior to the conduction of the tests. Emphsis ws given on voiding skips in trils in order to get bck to them lter. The order of the tests would lso be strictly followed: i.e., Prt I followed by Prts II, then Prts III nd IV. In the cued response test (Prt II), the subjects first red referent nd its description. Then they were sked to select, by putting mrk such s circle or x, one symbol from set of seven symbols they thought best represented the referent in question. Ech pge contined one referent description nd one set of symbols. Subjective certinties for their nswers using the 7-point rting scles (from very certin to very uncertin ) were likewise done. There were seven videophone referents tested on three sets of symbols rendering totl of twentyone trils. Referents Set 1 Set 2 Set 3 1. Cmer 2. Document Cmer 3. Hndsfree 4. Microphone 5. Selfview 6. Still Picture 7. Videophone Figure 1. The videophone referents nd the three sets of symbols used in the study (Böcker, 1993). 516

129 3. RESULTS AND DISCUSSION 3.1. Correct Associtions nd Subjective Certinties Percentges of correct ssocitions (hit rtes) nd subjective certinties in the cued-response tests re shown on Tble 1. Correct ssocition resulted when subject correctly mtched the referent in question to its right symbol. Correct responses were lbelled s 1 (hits) nd wrong responses s 0 (misses). Missing dt or no responses were initilly treted s 0 lso during this stge of the nlysis. Thus, the response dt were binomil percentges nd rcsine trnsformtions re wrrnted to ensure tht ssumptions in normlity were met. Such trnsformtions of percentges eliminte subjects s vrible leving only the independent vribles nd lowering the degrees of freedom. A repeted mesures fctoril design (multifctor ANOVA) ws then used to determine significnt differences in the hit rtes. Gmes- Howell post hoc tests were done to nlyse significnt min effects. TABLE 1. Hit Rtes in the Cued-Response Tests nd Subjective Certinties mong the three sets of grphicl symbols representing seven videophone functions, n = 48 per country. HITS Cmer Doc. Cmer Hndsfree Selfview Still Picture Countries Indonesi Mlysi Philippines Thilnd Sri Lnk Mens CERTAINTIES Cmer Doc. Cmer Hndsfree Selfview Still Picture Microphone Videophone Microphone Videophone Countries Indonesi Mlysi Philippines Thilnd Sri Lnk Mens

130 The hypotheses tested in this prt were if hit rtes would differ between sets nd between countries. The dt on Tble 1 showed tht between the three symbol sets (min effect) nd except for the hndsfree symbols, Set 1 symbols hd significntly higher hit rtes thn Sets 2 nd 3 in the seven referents tested (F(2, 30), p < 0.05). These results were expected prtly becuse Sets 1 nd 3 symbols were more concrete nd representtive of the referents they represented. With country s nother min effect, Indonesi, Mlysi, Philippines, nd Thilnd hd comprbly similr levels of hit rtes. Only Sri Lnk showed significntly lower hit rtes compred to the other countries in the symbols for cmer, document cmer, nd hndsfree (F(4, 15), p < 0.05) results tht would be discussed in detil lter. Regrding the certinty rtings, they were generlly higher for Sets 1 nd 3 compred to Set 2. The certinty scores of the symbols for microphone nd videophone were either very high (Sets 1 nd 3) or very low (Set 2). However, Kruskl-Wllis tests showed only few significnt differences between countries regrding their certinty rtings (Tble 2). Overll, when ISO 9186 nd ANSI Z535.3 comprehension levels were considered through hit rtes, most of the tested videophone symbols performed poorly. Combining country results, only seven of the 21 symbols reched ISO s required comprehension level of t lest 67%. These were Set 1 symbols for cmer (71%), document cmer (86%), microphone (84%), nd videophone (90%); Set 2 symbol for document cmer (68%); nd Set 3 symbols for microphone (83%), nd videophone (81%). If ANSI Z535.3 ws the bsis, only two of the symbols would ve met its 85% comprehension level (Set 1 symbols for document cmer, 86%, nd videophone, 90%). Thus, by looking merely t the hit rtes, the following could be stted. Very few of the symbols performed well regrdless of country, nd most of these were from Set 1. The symbols most poorly understood were bsiclly bstrct in their generl ttributes (Set 2). Moreover, when compring countries, only Sri Lnk seemed to differ significntly compred to the other countries. Notwithstnding tht some other culturl differences lso exist nd except for Sri Lnk, the four other countries lie firly close to ech other, with stndrds of living nd eduction quite similr s well. This cn thus prtilly explin the results but dmittedly will need more studies to be confirmed. However, by nlysing the errors or non-hit prmeters of the test procedures, more differences mong the symbols nd countries were discovered s discussed below. 518

131 TABLE 2. Kruskl-Wllis Tests for Subjective Certinties for ech symbol (n = 48 per country, df =4, p<0.05). Symbol Sets Set 1 Chi- Squre Asymp. Sig. Cmer Document Cmer Hndsfree Microphone Selfview Still Picture Videophone Set 2 Cmer Document Cmer Hndsfree Microphone Selfview Still Picture Videophone Set 3 Cmer Document Cmer Hndsfree Microphone Selfview Still Picture Videophone Confusion Mtrices nd Misses When symbols pper simultneously or s set nd the min tsk is to pick out which mong these symbols correspond to desired function or object, there re other possible outcomes side from hit. These re the so-clled non-hit outcomes or prmeters such s misses, confusions nd missing vlues (filure to respond or no nswer). The wrong nswers 519

132 during the cued response tests were nlysed using confusion mtrices (Tbles 4 to 8). In this prticulr prt of the study, tbultions were mde on the rtes of confusions the rte wherein ech symbol ws selected in the context of different referent. Confusions cn be clssified into two types: symmetric nd symmetric (Noln, 1989). The former occurs when subjects chose symbol x when presented with referent y, nd symbol y when presented with referent x. Symmetric confusions usully suggest visul or conceptul similrities. These were exemplified by the selfview / still picture (visul similrity) nd hndsfree / microphone symbols of Set 1 (conceptul similrity). Another cse of symmetric confusion due to visul similrities would be tht of the cmer / document cmer symbols of Set 3. These symmetric confusions were present in ll countries. Thus, the problem my not be culturlly-linked but design problem tht the symbols concerned were visully too similr. Moreover, ccording to bsic ergonomic design principles, these symbols would then be prime considertions for re-design or replcement to mke them more distinct from the other. TABLE 3. Confusion Mtrices, Indonesi, n = 48. Symbol Selected Referent Set 1 Set 2 Set 3 Presented Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Note: Figures by column represent confusion in percentges, when the symbol in ech column ws wrongly ssocited to the videophone referent (function) on first column. Figures by row represent distribution of misses per referent. Referents were presented one t time, nd symbols one set t time per referent. 520

133 TABLE 4. Confusion Mtrices, Mlysi, n = 48. Symbol Selected Referent Set 1 Set 2 Set 3 Presented Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Note: Figures by column represent confusion in percentges, when the symbol in ech column ws wrongly ssocited to the videophone referent (function) on first column. Figures by row represent distribution of misses per referent. Referents were presented one t time, nd symbols one set t time per referent. TABLE 5. Confusion Mtrices, Philippines, n = 48. Symbol Selected Referent Set 1 Set 2 Set 3 Presented Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Note: Figures by column represent confusion in percentges, when the symbol in ech column ws wrongly ssocited to the videophone referent (function) on first column. Figures by row represent distribution of misses per referent. Referents were presented one t time, nd symbols one set t time per referent. 521

134 TABLE 6. Confusion Mtrices, Thilnd, n = 48. Symbol Selected Referent Set 1 Set 2 Set 3 Presented Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Note: Figures by column represent confusion in percentges, when the symbol in ech column ws wrongly ssocited to the videophone referent (function) on first column. Figures by row represent distribution of misses per referent. Referents were presented one t time, nd symbols one set t time per referent. TABLE 7. Confusion Mtrices, Sri Lnk, n = 48. Symbol Selected Referent Set 1 Set 2 Set 3 Presented Cmer Doc. Cmer Hndsfree Microphone Selfview Still Picture Videophone Note: Figures by column represent confusion in percentges, when the symbol in ech column ws wrongly ssocited to the videophone referent (function) on first column. Figures by row represent distribution of misses per referent. Referents were presented one t time, nd symbols one set t time per referent. Asymmetric confusions occur when subjects simply chose the wrong symbol for given referent. For exmple, Tble 4 showed tht Set 1 s symbol for document cmer (1) ws wrongly selected by respondents from Indonesi when the referents cmer (10.4%), still picture (8.3%) nd videophone (2.1%) ws presented. Between sets, Set 2 clerly hd the most number of confusions bove this level nd Set 1 the lest in ll five countries. 522

135 Across symbol sets, selfview nd videophone symbols hd the highest instnces of being mistken s representing other functions. More differences were noted when the countries were compred bsed on symbol confusions. For exmple, while Thilnd hd the most instnces of confusing Set 1 s symbol for cmer thn the other four countries, it hd the lowest instnces of confusing Set 1 s videophone symbol for other functions. In symmetric confusions, the problem my lie in the vgueness or too much generlity of the symbol/s in question with regrds to the other referents. Thus, they cn esily be ssocited with severl referents. The results bove showed the countries with vrying ptterns of symmetric confusions. These could be useful in deciding which symbol (or referent) in ech country my need more explntions nd tests to void vgueness nd misconceptions of the functions being represented. Confusions re lso very useful indictors of the suitbility of the symbol nd my even complement the hit rtes. For exmple, the videophone symbols of Sets 1 nd 3 hd very similr hit rtes mong the countries. Nevertheless, when symbol confusion ws lso considered, Set 1 s version ws better becuse of its lesser number of confusions. With regrds to the microphone symbols wherein subjects from the five countries hd similr hit rtes lso, Set 3 s version performed best for Indonesi, Mlysi nd Sri Lnk (lower symbol confusions). In the cse of the Philippines nd Thilnd, Set 1 s microphone would be the better choice. Misses re the opposite of hits. These occur when the subject selected the wrong symbol for the referent being presented. Tbles 3 to 7 illustrte the misses (in percent) in rows in ccordnce to the referents it ws wrongly ssocited. The essentil thing with misses is tht its pttern of distribution under ech referent cn give n ide of the understndbility of the referent or function itself. For exmple, the referent hndsfree ws ssocited with lmost ll the symbols of in ll sets in quite high levels. This could suggest tht the concept of hndsfree might not be entirely cler to most of the subjects in ll countries Missing Vlues Missing vlues were instnces when some of the subjects gve no response or nswers during the cued response tests. Tble 8 illustrtes the distribution of the missing vlues cross countries for ech set. Between sets, Set 2 hd the highest instnces of missing vlues. Between countries, Thilnd hd the most instnces of missing vlues mong ll countries. Its missing vlues rnged from 2% to 23% cross ll 21 symbols. Sri Lnk ws the opposite with no missing vlues in the cued response test. When rryed with the other non-hit prmeters or errors, they re lso very importnt since they represent situtions wherein the prospective user plinly lcks the knowledge of which mong the symbols represent the desired function (referent). In ctul user scenrios, they cn be kin to non-use or under-use of the device. 523

136 TABLE 8. Missing Vlues or no nswers given in Cued Response Tests per set (n = 48 per country). Cmer Doc. Hndsfrephone Picture phone Micro- Still Video- Selfview Cmer Countries Indonesi Mlysi 2 Philippines Thilnd Sri Lnk 4. CONCLUSIONS In this study, three sets of videophone symbols designed nd tested in the west were tested mong subject groups from the est (Southest Asi). And when symbols re intended s prts of interfces of devices for interntionl use, different tests re usully needed. Usully, the different prts re designed to mesure initil ( priori) mening nd ppropriteness, ese of lerning nd remembering, s well s the probbility of confusion with other symbols. The cued response test simultes (though in limited mnner) rel videophone cll scenrios wherein the symbols re plced on the terminl with one function to be performed t time. It depicts situtions using the videophone the first few times with bits of informtion nd instructions given. Determintion of the hit rtes gives the level of ssocition of the symbol to its intended referent or function. However, when symbols pper simultneously, the hit rtes cnnot revel the dynmics of how ech symbol cn ffect the other s level comprehension nd ssocition to its referent. Thus, nlyses of the errors or non-hit prmeters re crucil. Confusions re instnces when symbols re selected under the wrong referent contexts. Together with hit rtes, they cn give n ide on symbol s distinctness from other symbols under different referent contexts s well s the different ptterns of their interctions in different groups. Subjective certinties re lso importnt. They reflect the subjects degree of confidence or confusion with their nswers. Other prmeters lso exist, which re not discussed but re lso helpful in evluting grphicl symbols. For exmple, preferences generlly indicte the esthetics. They re useful especilly when considering which mong symbols with similr hit rtes re more ppeling for the subjects. Lstly, this study ws just one prt of n interntionl project. It ttempted to demonstrte how multiple indices, including those tht nlyse non-hits cn be helpful in evluting symbols cross different user groups. Different ptterns in errors between countries were shown, but dmittedly, deeper nlyses re still needed to convert such observtions into more prcticl guides towrds designing symbols cross diverse user groups issues tht would be ddressed by the succeeding prts of the project. 524

137 Altogether, the results of this study supports the view tht nlyses of errors or nonhit prmeters such s confusions, misses, nd missing vlues re invluble in properly evluting grphicl symbols when they pper simultneously or in groups. They enble the tester to see other often subtle but importnt differences on how different users perceive nd understnd symbols. These re importnt not only in properly interpreting test results of different user groups but lso in formulting instructions nd other ids in lerning to use new products fster nd more stisfctorily. Acknowledgements To Mr. Mrtin Böcker for his invluble support in cquiring the videophone symbols used in this study nd in shring his thoughts nd expertise throughout. To KFB (Swedish Communictions Bord) for the finncil nd other logisticl ssistnce. REFERENCES Americn Ntionl Stndrds Institute (ANSI). (1991). Accredited stndrd on sfety colors, signs symbols, lbels nd tgs (Stndrd No. ANSI Z ). Wshington, DC, USA: Ntionl Electricl Mnufcturers Assocition. Böcker, M. (1993). A multiple index pproch for the evlution of pictogrms. In Proceedings of the 14th Interntionl Symposium on Humn Fctors in Telecommunictions, Drmstdt, Germny (pp ). Heidelberg, Germny: R. v. Decker's Verlg, G. Schenck GmbH. Europen Telecommunictions Stndrds Institute (ETSI). (1993). Technicl Report - Humn Fctors (HF): Results of n evlution study of pictogrms for point-to-point videotelephony. Sophi Antipolis, Frnce: ETSI. Interntionl Telecommunictions Union (ITU). (1995) Recommendtion F.910 (02/95) - Procedures for designing, evluting nd selecting symbols, pictogrms nd icons. Genev: ITU. Mgyr, R. L. (1990). Assessing icon ppropriteness nd icon discriminbility with pired comprison testing procedure. In Countdown to the 21st Century. Proceedings of the Humn Fctors Society 34th Annul Meeting, Orlndo, Florid (pp ). Snt Monic, CA, USA: Humn Fctors Society. Noln, P. R. (1989). Designing screen icons: rnking nd mtching studies. In Proceedings of the Humn Fctors Society 33rd Annul Meeting (pp ). Snt Monic, CA, USA: Humn Fctors Society. Tudor, L. G. (1994). The genertion nd testing of cndidte telecommunictions icons, In Blumenthl, B., Gornostev, J., nd Unger, C. (Eds.), Proceedings of the Est-West Interntionl Conference in HCI, St. Petersburg, Russi (pp ). Moscow, Russi: Interntionl Centre for Scientific nd Technicl Informtion (ICSTI). 525

138 Vor, P., Helnder, M., Swede, H., & Wilson, J. (1991). Developing guidelines for symbol design: comprison of methodologies. In Proceedings of Interfce 1991 (pp. 6-11). Dyton, OH, USA: University of Dyton. Webb, J. M., Sorenson, P. F., & Lyons, N. P., 1989, An empiricl pproch to the evlution of icons. SIGCHI Bulletin, 21, Zwg, H. L. G. (1989). Predicting the public's comprehension of grphic symbols. In Megw, E. D. (Ed.), Contemporry Ergonomics 1989 (pp ). London, UK: Tylor & Frncis. 526

139 PAPER 4 TESTING VIDEOPHONE GRAPHICAL SYMBOLS IN SOUTHEAST ASIA

140

141 TESTING VIDEOPHONE GRAPHICAL SYMBOLS IN SOUTHEAST ASIA Pimonte, D. P. T. 1, Abeyseker, J. D. A. 1 nd Ohlsson, K. 2 1 Division of Industril Ergonomics, 2 Division of Engineering Psychology, Deprtment of Humn Work Sciences, Luleå University of Technology, Sweden Introduction Southest Asi is region chrcterized by diversity in lnguge nd culture. Although it is slowly recovering from the recent economic crisis, it still remins s one of the most dynmic regions in the world. Economic trdes nd technology trnsfers from the west still bound in most of its countries. One plpble sign is the continued rise in the use of western products. With these lso cme the use of grphicl symbols (icons nd pictogrms) s common mode of user interfce of the products. Esy lernbility, less spce requirements nd being non-text dependent re just few of the resons why grphicl symbols hve become mjor mens of presenting informtion nd functions especilly in trvel, communiction nd consumer products (Zwg, 1989 nd Horton, 1994). In order to elicit the bove benefits of using grphicl symbols, they must be tested cross their intended user groups. Symbols re often designed nd tested in the west but trgeted for interntionl use. Thus, the extent of their usbility nd cceptbility mong user groups with different lnguges nd cultures from those of the west cn be questionble. Studies on how icons nd pictogrms in Asi re still sprse. This pper imed to study how symbols designed nd tested in Europe fred mong subjects from different countries in Southest Asi. It ws prt of n interntionl study on grphic symbols involving countries from Southest Asi (Pimonte, 1998). Specificlly, videotelephone symbols designed nd tested by the Europen Telecommuniction Stndrds Institute (ETSI) (Böcker, 1993) were evluted using subjects from five Southest Asin countries. Methods nd Mterils Three sets of videophone symbols were derived studies by Böcker (1993) for the Europen Telecommunictions Stndrds Institute (ETSI), including the set (Set 1) eventully proposed by ETSI s the stndrd (Figure 1). The symbols were tested in the form of questionnires. Ech symbol set contined seven symbols representing seven videophone functions. 793

142 Testing of the subjects were done round Metro-Mnil, Philippines, Bngkok, Thilnd, Bndung, Indonesi, Srwk, Mlysi, nd Sri Lnk, nd in were nd in smll groups in ech country, lsting for bout minutes. Subjects were rndomly given one of three versions of the questionnires. The subjects were shown n illustrtion of videophone. Its generl functions were then discussed. Afterwrds, instructions were given on how to go bout the different test prts. Questions were entertined prior to the dministrtion of the tests. Emphsis ws given on voiding omissions in trils in order to get bck to them lter on. In the spontneous identifiction test (Prt One), ech pge contined one set of the seven videophone symbols, for totl of three pges for the three symbol sets. On spces provided for, they would write wht videophone function they thought ws represented by ech symbol. They were lso sked to rte the level of certinty for ech of their nswers using seven-point rting scle (from very certin to very uncertin ). Prt Two ws cued response test ptterned fter method developed by Böcker (1993) in testing simultneously ppering symbols. The subjects first red referent nd its description. Then they hd to choose one symbol from set of seven symbols which they thought best represented the referent in question. Ech pge contined one referent-description nd one set of symbols. They were lso sked to rte their certinties for their nswers using the 7-point rting scles. There were seven videophone referents tested on three sets of symbols rendering totl of twenty-one trils in Prt Two. Results nd Discussion Two hundred forty-eight subjects (127 mles nd 113 femles) from the five Asin countries cted s respondents. The men ge ws yers (SD=6.08, Rnge=17-52 yers). There were forty-eight subjects per country with equl numbers of students nd employees. In both Prts One nd Two, correct responses resulted when subject correctly identified (Prts One) or mtched (Prt Two) the referent in question to its right symbol. Correct responses were lbeled s 1 (hits) nd wrong responses s 0 (misses). Missing dt or no responses were initilly treted s lso 0 during this stge of the nlysis. 794

143 Referents Set 1 Set 2 Set 3 Cmer Document Cmer Hndsfree Microphone Selfview Still Picture Videophone FIGURE 1. Grphicl symbols used in the present study s bsed on the studies by Böcker (1993) for the Europen Telecommunictions Stndrds Institute. The response dt were thus bsiclly binomil percentges nd rcsine trnsformtions re wrrnted to ensure tht ssumptions in normlity were met. Such trnsformtions of percentges eliminte subjects s vrible leving only the independent vribles nd lowering the degrees of freedom. A repeted mesures fctoril design (multi-fctor ANOVA) with Tukey s honestly significnt differences s post hoc tests were done to determine ny significnt differences in the hit rtes bsed on test type, subject group nd symbol set. Kruskl-Wllis tests were likewise done to compre certinty rtings. 795

144 Between tests, correct responses in the cued response tests (Prt Two) were significntly higher (t F(1, 111), p<0.05) thn in the spontneous identifiction tests (Prt One), except for the symbols for microphone. Between symbol sets, Set 1 hd significntly higher hits in both tests for ll referents except for document cmer nd hndsfree (F(2, 111), p<0.05). On the other hnd, Set 2 grnered the lowest hits in both Prts One nd Two of the test. Furthermore, only the symbols (of Sets 1 nd 3) for referents microphone nd videophone grnered hit rtes of t lest 67% in Prt One. Altogether in both tests nd cross ll symbol sets, the highest hit rtes were for cmer, microphone nd videophone while the document cmer, hndsfree, selfview nd still picture hd the lowest hit rtes in ll countries. Compring countries, there were significnt differences (F(4, 111), p<0.05) in hit rtes for symbols for cmer, document cmer, hndsfree. In these symbols, Sri Lnkn subjects generlly hd the lowest hits compred to the other four countries. Certinty rtings were mostly higher for Sets 1 nd 3 compred to Set 2 in both Prt One nd Prt Two. Certinty rtings of the symbols for microphone nd videophone were either very high (Sets 1 nd 3) or very low (Set 2) in both Prts One nd Two. No significnt differences were noted when subjects were compred bsed on occuption (student vs. employees/professionls). Confusions cn be clssified into two types: symmetric nd symmetric (Noln, 1989). The former occurs when subjects chose symbol x when presented with referent y, nd symbol y when presented with referent x. Symmetric confusions usully suggest visul or conceptul similrities. Asymmetric confusions occur when subjects simply chose the wrong symbol for given referent. Tbultions mde on the rtes of confusions showed more differences between countries. For exmple, while Thilnd hd the most instnces of confusing Set 1 s symbol for cmer thn the other four countries, it hd the lowest instnces of confusing Set 1 s videophone symbol for other functions. Confusions re very useful indictors of the suitbility of the symbol nd my even complement the hit rtes. For exmple, the videophone symbols of Sets 1 nd 3 hd very similr hit rtes mong the countries. Nevertheless, when symbol confusion ws lso considered, Set 1 s version ws better becuse of its lesser number of confusions. With regrds to the microphone symbols wherein subjects from the five countries hd similr hit rtes lso, Set 3 s version performed best for Indonesi, Mlysi nd Sri Lnk (lower symbol confusions). In the cse of the Philippines nd Thilnd subjects, Set 1 s microphone would be the better choice. Lstly, missing vlues were instnces when some of the subjects gve no response or nswers during the cued response tests. Between countries, Thilnd hd the most instnces of missing vlues mong ll countries. Its missing vlues rnged from 2% to 23% cross ll 21 symbols. Sri Lnk ws the opposite with no missing vlues in the cued response test. Missing vlues or subjects giving no nswers s prmeter is very good guge of situtions wherein the subjects did not know the nswer becuse he/she couldn t understnd the referent or the symbols presented. In ctul user scenrios, this would be kin to the user not knowing which control would cuse the trget function (Böcker,1993). Altogether, some both similrities nd differences were noted regrding symbol understnding mong Southest Asin subjects. The results in the spontneous identifiction nd cued response tests in testing videophone symbols showed very low rtes of correct identifiction nd even correct ssocition of the symbols cross ll countries. Differences between countries lie mostly in the levels of confusing the symbols from ech other nd their missing vlues. Both results showed the potentil difficulties encountered by people from this region of recognizing nd using these symbols. However, it is still believed tht grphicl symbols undenibly re still very useful. When conditions 796

145 wrrnt tht they be used lone or s primry interfce, user testing, dequte informtion, trining s well s other help procedures become very importnt. References Böcker, M. (1993). A multiple index pproch for the evlution of pictogrms, In: Proceedings of the 14th Interntionl Symposium on Humn Fctors in Telecommunictions, Drmstdt, Germny, My 11-14, Heidelberg: R. v. Decker's Verlg, G. Schenck GmbH, pp Horton, W. (1994). The Icon Book Visul symbols for computer systems nd documenttion, New York: Wiley nd Sons. Noln, P. R. (1989). Designing screen icons: rnking nd mtching studies. In Proceedings of the Humn Fctors Society 33rd Annul Meeting, Snt Monic: Humn Fctors, pp Pimonte, D. P. T. (1998) A User-bsed Evlution of Cndidte Telecommuniction Icons, Industril Ergonomics Division nd Engineering Psychology Division, Humn Work Sciences Deprtment, Luleå University of Technology-Sweden, ISSN: , ISRN:LTU-LIC, 98/12-SE, Luleå: Luleå University of Technology. Zwg, H. L. G. (1989), Predicting the public s comprehension of grphic symbols. In Megw, E. D. (Ed.): Contemporry Ergonomics, pp

146

147 PAPER 5 AN EMPIRICAL EVALUATION OF VIDEOPHONE SYMBOLS: AN INTERNATIONAL STUDY

148

149 AN EMPIRICAL EVALUATION OF VIDEOPHONE SYMBOLS: AN INTERNATIONAL STUDY Pimonte, D. P. T. 1, Ohlsson, K. 2, nd Abeyseker, J. D. A. 1 1 Division of Industril Ergonomics, 2 Division of Engineering Psychology, Deprtment of Humn Work Sciences, Luleå University of Technology, Sweden Introduction The widespred use of grphicl symbols in the light of the current economic globlistion is undenible. Symbols such s icons or pictogrms s interfces in modern technologicl devices hve become very common. Although generlly designed in western countries, their use cn be found worldwide from computers to udio-visul pplinces nd other similr devices. Their use offers severl potentil dvntges. They require minimum spce, re esy nd quick to recognise, nd re non-text dependent. The use of grphicl symbols, however, is not devoid of problems. Foremost mong these is tht some symbols re simply not understood well (Brelsford, Woglter nd Scoggins, 1994). The bove dvntges were cited s evident when the concepts being represented were well understood or concrete, nd not when ides become more bstrct. Regrding older people, Dvis, Hines, Norris nd Wilson (1998) cited the studies of Esterby nd Hkiel (1981) showing tht lthough symbols re esily recognizble, understnding their mening re generlly poorer compred to younger people. One bsic principle of effective UI design is to involve the potentil users in the design nd evlution stges especilly when the user groups re quite diverse. Thus, empiricl tests re importnt to properly evlute the symbols on their usbility especilly cross different cultures. This study ws prt of n interntionl project on evluting telecommuniction symbols (Pimonte, Ohlsson nd Abeyseker, 1999b). One of the mjor objectives ws to test different grphicl symbols (in this cse, those of the videophone), designed nd tested in Western Europe using subjects from different subject groups from Asi nd the United Sttes. The elderly s specil group ws lso included. The previous pper delt with the study using subjects from five Southest Asin countries (Pimonte, Abeyseker, nd Ohlsson, 1999). This pper would discuss the results involving U.S. nd Finnish elderly subjects nd compring them with the Asin results s well. Methodology The mterils (symbols nd questionnires) used in the previous pper were lso used in this study (Pimonte et l, 1999 nd b). However, side from the spontneous identifiction nd cued response tests, symbol nd set preferences were lso tested. Except for the spontneous identifiction tests (Prt One), the whole method ws bsed on the multiple index pproch (MIA) developed by Böcker (1993) with the Europen Telecommunictions Stndrds Institute (ETSI). 798

150 In the spontneous identifiction test (Prt One), ech pge contined one set of the seven videophone symbols, for totl of three pges for the three symbol sets. On spces provided for, they would write wht videophone function they thought ws represented by ech symbol. They were lso sked to rte the level of certinty for ech of their nswers using seven-point rting scle (from very certin to very uncertin ). In the cued response test (Prt Two), the subjects first red referent nd its description. Then they hd to choose one symbol from set of seven symbols which they thought best represented the referent in question. Ech pge contined one referent-description nd one set of symbols. They were lso sked to rte their certinties for their nswers using the 7-point rting scles. There were seven videophone referents tested on three sets of symbols rendering totl of twenty-one trils in Prt Two. In the lst prt, the three symbols (one for ech set) together with the referent they represented were shown. The subjects hd to choose one symbol they thought best represent tht referent. Next, with the three symbols sets presented together, they hd to choose the set they prefer most. In ll, the subjects would choose seven symbols to represent the seven referents nd one symbol set they preferred most. Results nd Discussion For purposes of extended comprisons, subjects from the United Sttes were included, using the sme method in the Southest Asin studies (Pimonte et l, 1999). The nineteen subjects were students nd employees in the University of Centrl Florid. The ge rnge ws yers with men of 22.6 yers (SD=5.24). All of them hve used computers nd computer-relted products t home nd t work, with 78.9% (n = 15) using both grphicsbsed nd DOS-bsed progrms. Similr to the previous study, the US subjects performed poorly in Prt One, even filing to identify the hndsfree symbols in ll 3 sets. The still picture symbols filed to score hits bove 67% in both Prt One nd Prt Two. In contrst, the symbols for cmer, microphone nd videophone hd high recognition rtes regrdless of test type nd set. Set 1 hd the most number of symbols with recognition rtes bove 67% nd Set 2 with the lest. Certinty rtings generlly were higher when the hits were higher s well. Aside from correct identifiction nd ssocitions, other prmeters were lso mesured. These were confusion, missing vlues, symbol nd set preferences. Regrding preferences, the US subjects preferred five of seven of Set 1 s symbols. These results were similr to the Southest Asin study. However, they differ from the ltter group by eqully preferring Sets 1 nd 3 s group. One importnt fctor tht leds subjects or potentil users to their choice of symbols is esy recognition. Set 3 s hit rtes were generlly comprble to Set 1. Hence, it ws not surprising for preference to be similr s well. In some prts of Northern Finlnd, videophone-bsed services for the elderly re being tested. This ws one of the mjor resons why the current study ws lso tested mong group of elderly subjects. The respondents in the tests were twenty-nine (29) subjects from Finlnd (14 women nd 15 men). The women were 61 to 81 yers old (men ge: 69.6 yers) nd the men 60 to 88 yers old (men ge: 70.9 yers). The men ge for the whole group ws 70.3 yers. Compred to the subjects from Southest Asi nd even the USA, the Finnish elderly filed to do Prt One. Mjority of the subjects climed tht the tsks were either too difficult or too complicted. When compred to younger subjects (from Asi nd the US), the elderly subjects hd much lower hit rtes in ll symbols of the three sets used. Lstly, in contrst to the other groups lso, the preference tests reveled the elderly fvoring Set 3 to Sets 1 nd 2. They commented tht Set 3 s symbols were more concrete nd esily recognizble thn the others. 799

151 The trend of demogrphic evolution shows tht the Europen popultion, in prticulr, is one tht is becoming old. This trend will hve implictions on the socil structure, where respect for n independent living should be tken into considertion. It should tking into ccount the new services nd products just vilble or those under development, such s telelrms nd teleservices to ssist people t home or elsewhere. Understnding the humn fctors of ging cn led to computer designs tht will fcilitte ccess by the elderly. Benefits include incresed ccess of the society to the elderly for their experience, incresed prticiption of the elderly in society through communiction networks, nd improved chnces for productive employment of the elderly. (Shneidermn, 1992). In this cse, the symbols were intended to improve the usbility of product (i.e. videophone). However, the results showed tht the opposite might occur mong the elderly. Only by conscious nd ctive prticiption of the elderly in designing nd developing modern products will their needs be truly met. As whole, empiricl tests using multiple indices re importnt to properly evlute the symbols on their usbility especilly cross different user groups. The evlution method should lso simulte ctul usge scenrios to determine the suitbility of ech symbol lone, but together with the other symbols s well. This cn be exemplified by the elderly nd US subject groups who chose symbol sets quite different from the Asins. Hit rte is n importnt prmeter, but must be tested under different user scenrios (spontneous identifiction nd cued responses) to determine if the symbols indeed re understood with nd without cues. At the sme time hits rtes lone re not enough. Subjective certinties, flse lrms (confusions), preferences, nd even missing vlues re eqully useful fctors needed to mke deeper nlysis. They enble the tester to see other often subtle but importnt differences (i.e. subjective bises, confusing symbols to other referents nd symbols) on how users perceive nd understnd symbols. Acknowledgements To Mr. Mrtin Böcker, of Siemen s-nixdorf nd ETSI Humn Fctors Group, for his generosity not only in shring his resources on icons nd pictogrms, but more so for unselfishly shring his insights nd ides bout the study. To the Ntionl Communictions Reserch Bord of Sweden (Kommuniktions Forsknings Beredningen, KFB) for the finncil nd logisticl support References Brelsford, J. W., Woglter, M. S., nd Scoggins, J. A., (1994). Enhncing comprehension nd retention of sfety-relted pictorils, In Proceedings of the Humn Fctors Ergonomics Society 38th Annul Meeting, pp Böcker, M. (1993). A multiple index pproch for the evlution of pictogrms, In: Proceedings of the 14th Interntionl Symposium on Humn Fctors in Telecommunictions, Drmstdt, Germny, My 11-14, Heidelberg: R. v. Decker's Verlg, G. Schenck GmbH, pp Dvies, S., Hines, H., Norris, B., nd Wilson, J., (1998). Sfety pictogrms: re they getting the messge cross? Applied Ergonomics, 29, 1, Esterby, R. S., nd Hkiel, S. R., (1981). Field testing of consumer sfety signs: the comprehension of pictorilly presented messges. Applied Ergonomics, 12, 3, Horton, W. (1994). The Icon Book Visul symbols for computer systems nd documenttion, New York: Wiley nd Sons. 800

152 Pimonte, D. P. T., Abeyseker, J. D. A., nd Ohlsson, K. (1999). Testing videophone grphicl symbols in Southest Asi. In Proc. 8th Int. Conference on Humn-Computer Interction (HCI Interntionl '99, Munich, Germny, August 22-27, 1999). Pimonte, D. P. T., Ohlsson, K., Abeyseker, J. D. A. (1999b). Finl Project Report: Across the ses-auser - bsed evlution of cndidte telecommuniction icons (Sponsored by the Communictions Reserch Bord of Sweden, KFB). Luleå: Luleå University Press (in press). Shneidermn, B. (1992). Designing the user interfce Strtegies for effective humncomputer interction (2nd edition). New York: Addison-Wesley. 801

153 PAPER 6 Understnding Smll Grphicl Symbols: A Cross-Culturl Study

154

155 Understnding Smll Grphicl Symbols: A Cross-Culturl Study D. Pul T. Pimonte 1, John D. A. Abeyseker 1 nd Kjell Ohlsson 2 1 Division of Industril Ergonomics, Dept. of Humn Work Sciences, Luleå University of Technology, Luleå, Sweden 2 Division of Humn Work Sciences, Linköping University, Linköping, Sweden Abstrct Grphicl symbols such s icons or pictogrms s interfces in modern technologicl devices hve become quite common. Although generlly designed in western countries, their use cn be found worldwide from udio-visul pplinces to computers nd computer-relted devices. A bsic ergonomic tenet is to involve the potentil users in the design nd evlution stges especilly when the user groups re quite diverse. This study ws prt of n interntionl project on evluting telecommuniction symbols. One of the mjor objectives ws to test different grphicl symbols (of the videophone), designed, nd tested in Western Europe using subject groups from Asi, Europe, nd the United Sttes. This pper would discuss the mjor portions of the tests involving U.S. nd Swedish subjects. Performnce indices used were hits, certinties, confusions, nd semntic differentil rtings. They were useful in nlysing how the symbols were recognised, confused, nd perceived by different subject groups. They lso helped detect differences between groups which otherwise seemed to hve similr test results. Implictions of the use of such multiple performnce indices in symbol design nd testing mong diverse groups would lso be discussed. Keywords: Grphicl symbols, icons, pictogrms, cross-culturl studies, evlution, semntic differentil testing 1 Introduction The ubiquitous nture nd pplicbility of the current computer nd informtion technology cn be felt worldwide. Innumerble tools nd equipment in schools, homes, nd workplces hve become computer-bsed. From the lowly flt irons, ovens nd telephones, to different udio-video pplinces nd high-tech desktop nd notebook computers, ll of these hve become equipped with more complex functions. Consequently, one needs to lern how to mnipulte grphic displys nd buttons to vil of these dded functions. In the midst of these complicted mchines nd the humns ttempting to utilise re the grphicl symbols. In generl, symbol is ny grphicl chrcter or other representtion. It is intended to () stnd for something else, (b) communicte use for n object/structure, or (c) communicte wht should or should not be done t given time or loction (Strmler 1993). Grphicl symbols, in turn, usully pertin to terms like icons, nd pictogrms or pictoril symbols (Böcker, 1993). The former refers to symbols tht re simple, concrete nd usully selfexplntory of the ides, objects, or functions they represent (Wood nd Wood 1987). The ltter re usully more bstrct, conveying messges by nlogy or symbolism. Thus, pictogrms require certin lerning processes to be understood. 1

156 In ddition, when one tlks of computer systems nd documenttion, Horton (1994) described icons s the smll pictoril symbols used on the computer menus, windows, nd screens. They represent the cpbilities of the computer system. They cn be ctivted nd bring forth these cpbilities into ction. By using powerful microprocessors or microchips, lmost ny device cn now even be equipped with dditionl (nd usully more complex) functions. Grphicl symbols re thus idel s interfce. They use very little spce, re nontext dependent but cn convey lots of informtion (Mguire, 1985). They help the products go globl. Thus, the rpid prolifertion of complicted nd multifunctionl devices nd their symbols is complemented by the culturl diversity of the intended users. The importnt question then, is whether symbols re relly understood the wy they should be cross groups of vrying cultures. Only during the lst few yers hs reserch gined ground on this field of culturl differences in user perception (Resnick et l. 1997, Lin 1999). This study ws prt of n interntionl project on evluting telecommuniction symbols. One of the mjor objectives ws to test different grphicl symbols designed nd tested in Western Europe using subjects from different subject groups from Asi, Europe nd the United Sttes. This pper would report on the prts involving Europen (Swedish) nd US (Americn) subjects. Method Eighty-eight university students nd employees of smll nd lrge compnies from Sweden (48, i.e. 24 mles nd 24 femles) nd the United Sttes (40, i.e. 21 mles nd 19 femles) prticipted in the study. The test method used three sets of icons nd pictogrms representing seven referents or functions of the videotelephone (Figure 1). Both referents nd pictogrms were from studies of the Europen Telecommunictions Stndrds Institute (ETSI) with one of the sets (Set 1) recommended by ETSI s the stndrd. The test ws in the form of questionnires. Ech questionnire hd four prts: symbol identifiction tests, cued response tests, semntic differentil rtings, nd symbol nd symbol set preferences. Cmer Document Cmer Hndsfree Selfview Still Picture Microphone Video- Phone SET 1 [1) [4) [7) [10) [13) [16) [19) SET 2 [2] [5] [8] [11] [14] [17] [20] SET 3 [3] [6] [9] [12] [15] [18] [21] FIGURE 1. The seven videophone referents nd the three sets of symbols used in the study. 2

157 The subjects were rndomly given one of three versions of the questionnires. An illustrtion of videophone ws shown nd its generl functions were then discussed. Afterwrds, instructions were given on how to go bout the different test prts. Questions were entertined prior to the conduction of the tests. Emphsis ws given on voiding omissions in trils in order to get bck to them lter. The order of the tests would lso be strictly followed; i.e. Prt One followed by Prts Two, Three, then Four. In the spontneous identifiction test (Prt One), ech pge contined one set of videophone symbols rrnged in rndom order. A subject s tsk ws to write on spces provided wht videophone function he/she thought ech symbol represented. The sme tsks were to be repeted for the two other symbol sets. In the cued response test Prt Two), the subjects were to red referent nd its description. They then chose one symbol from set of seven symbols they thought best represented the referent in question. Ech pge contined one referent description nd one set of symbols. In both Prts One nd Two, the subjects were lso sked to rte their certinties for their nswers using the 7-point rting scles. In the third prt, five semntic scles were used to evlute ech symbol together with its correct referent, for totl of twenty-one SDTs. In Prt Four, the three symbols (one for ech set) together with the referent they represented were shown. The subjects hd to choose one symbol they thought best represent tht referent. Next, with the three symbols sets presented together, they hd to choose the set they prefer most. In ll, the subjects would choose seven symbols to represent the seven referents nd one symbol set they preferred most. Results nd discussion Tble 1 contins the mens of the correct recognition scores (hits) nd the mens certinty rtings of the Swedish nd US subjects for Prts One nd Two using the three symbol sets. Regrding identifiction tests (Prts One nd Two), Set 1 symbols generlly hd the higher hit rtes in both countries. However, both countries performed poorly in the spontneous identifiction tests (Prt One). For exmple, subjects from both countries mnged to ttin the Orgniztion for Interntionl Stndrdiztion s ISO 3864 (ISO, 1984) minimum correct recognition rte of 66.7% in only 5 of the 21 symbols tested. Compred to some erlier studies using the sme symbol sets, lthough the Swedish nd Americn subjects fred quite better thn their Asin counterprts (Pimonte 1998 nd 1999) especilly in Prt One, the scores were still mostly below the criticl 66.7%. Spontneous identifiction tests simulte, lbeit, in limited mnner, the first-time encounter of n interfce. The poor results thus strongly suggest tht without sufficient lerning opportunities nd lerning ids, symbols my be hrder to understnd by lmost nyone thn previously thought. However, when trying to study possible culturl influences in symbol evlution nd understnding, levels of correct identifiction re not enough. Other prmeters re essentil (Noln 1989, Lin 1999). This ws shown by the results of the other prmeters such s certinties (Tble 1), confusions (Tbles 2-4) nd semntic differentil scles (Tble 5). Compred to the Americn subjects, Swedish subjects tended to give lower certinty rtings especilly during the spontneous identifiction tests. This cn suggest certin subject bis. Confusion mtrices showed some differences between the symbol sets nd country groupings. Agin, Set 1 hd the lest number of significnt confusions in both countries. However in Set 1, Swedish subjects tend to hve more problems confusing the hndsfree function s represented by the symbols for videophone (9) nd microphone (9), while Americns confused the former more with the microphone function (7) (Tble 2). In ddition, Swedish group showed symmetricl confusions between the selfview nd still picture symbols of Set 1 (11, 13). These were not observed with the US subjects. 3

158 Confusions re very importnt in design considertions. They show which mong the symbols need to be chnged or re-designed or which mong the functions re metphoriclly relted tht trining needs to be emphsised during the erly prt of using the symbols. Semntic scles hve been shown to be very useful in studying how symbols re perceived (Lin 1999). The scles used in this study were bsed mostly on the studies by Lin (1999) nd Dewr (1977). In the semntic scles, gin, the Swedish group s whole gve lower men semntic rtings compred to the Americn group. Although, there is high correltion between the hit rtes nd ll the semntic scles, both the Swedish nd US subjects hd highest rtings for the symbols meningfulness nd simplicity, with the ltter group putting lso prime importnce to concreteness. As mentioned bove, the differences in ptterns of rtings suggest tht some subject bis occur between the two groups, which my be culturlly-linked nd cn help in determining which spects of symbol design nd usge my be more helpful (ex. instructions, lerning ids, etc.). Awreness of such subject bis nd their implictions re importnt on how one interprets the test results. For exmple, the Swedish subjects tended to be more conservtive on their rtings compred to the Americns. However, this does not necessrily men lower level of correct symbol perception (s the hits rte results showed). Admittedly though, the bove findings need to be nlysed more deeply to be ble to concretely trnsform such differences into prcticl pplictions of symbol design nd testing. Acknowledgements To Mr. Mrtin Böcker for his invluble support in cquiring the videophone symbols used in this study nd in shring his thoughts nd expertise throughout. To Prof. Michel Kpln of Centrl University of Florid, for shring his precious time doing prts of the tests in USA. To KFB (KommuniktionsForskningsBeredningen - Swedish Communictions Bord) for the finncil nd other logisticl ssistnce. References Böcker, M., A multiple index pproch for the evlution of pictogrms. Proceedings of the 14th Interntionl Symposium on Humn Fctors in Telecommunictions. R. v. Decker's Verlg, G. Schenck GmbH, Heidelberg, pp Böcker, M., Symbol design. In A Focus Seminr in Symbol Design nd Evlution: Forum for Corporte Usbility within Ericsson. Stockholm, Sweden. Dewr, R.E. nd Ells, J.G., The semntic differentil s n index of trffic sign perception nd comprehension. Humn Fctors, 19 (2): Horton, W., The Icon Book - Visul Symbols for Computer Systems nd Documenttion. John Wiley nd Sons, New York, 417 pp. Interntionl Stndrds Orgniztion (ISO), Interntionl stndrd for sfety colours nd sfety signs: ISO Genev, Switzerlnd. Lin, R., Culturl differences in icon recognition. In: G. Slvendy, M. Smith nd R. Koubek (Eds.), Design of Computing Systems Volume I; Proceedings of the 8th Interntionl Conference on Humn-Computer Interction,. Amsterdm: Elsevier, pp

159 Mguire, M.C., A review of humn fctors guidelines nd techniques for the design of grphicl humn-computer interfces. Computers nd Grphics, 9 (3): Noln, P.R., Designing screen icons: rnking nd mtching studies. In Proceedings of the Humn Fctors Society 33rd Annul Meeting, Humn Fctors, Snt Monic, pp Pimonte, D.P.T., A User-bsed Evlution of Cndidte Telecommuniction Icons, Licentite Thesis. ISSN: , ISRN: LTU LIC -- 98/12 -- SE Divisions of Industril Ergonomics nd Engineering Psychology, Deprtment of Humn Work Sciences, Luleå University of Technology, Luleå, 90 pp. Pimonte, D.P.T., The relevnce of error nlysis in grphicl symbols evlution. Interntionl Journl of Occuptionl Sfety nd Ergonomics (in press). Resnick, M.L., Znotti, A. nd Jcko, A., Culturl differences in the perception of responsibility for child sfety. In Proceedings of the Humn Fctors Society 39th Annul Meeting, Humn Fctors, St. Monic, pp Strmler, J.H. Jr., The Dictionry for Humn Fctors/Ergonomics. CRC Press, London, 413 pp. Wood, W.T. nd Wood, S.K., Icons in everydy life. Socil Ergonomics nd Stress Aspects of Work with Computers,

160 TABLE 1. Mens correct recognition scores (hits) nd certinty rtings for Prts One nd Two, Swedish nd US subjects (Sweden, n=48, USA, n=40). PART ONE (Spontneous Identifiction) PART TWO (Cued-Response) SWEDEN USA SWEDEN USA Hits Cert. Hits Cert. Hits Cert. Hits Cert. [1] Cmer [2] Cmer [3] Cmer [4] Doc. Cmer [5] Doc. Cmer [6] Doc. Cmer [7] Hndsfree [8] Hndsfree [9] Hndsfree [10] Microphone [11] Microphone [12] Microphone [13] Selfview [14] Selfview [15] Selfview [16] Still Picture [17] Still Picture [18] Still Picture [19] Videophone [20] Videophone [21] Videophone Overll Mens

161 TABLE 2. Prt Two confusion mtrices for Set 1, per country. SWEDEN USA SET 1 HF SE SP CA VP DC MP SET 1 HF SE SP CA VP DC MP Hndsfree - 7 Hndsfree Selfview Selfview Still Picture 11-3 Still Picture 1 - Cmer Cmer Videophone Videophone Doc. Cmer Doc. Cmer Microphone 9 - Microphone 7 - Missing Missing 2 TABLE 3. Prt Two confusion mtrices for Set 2, per country. SWEDEN USA SET 2 HF SE SP CA VP DC MP SET 2 HF SE SP CA VP DC MP Hndsfree - 17 Hndsfree Selfview Selfview Still Picture Still Picture Cmer Cmer Videophone Videophone Doc. Cmer Doc. Cmer Microphone Microphone Missing Missing TABLE 4. Prt Two confusion mtrices for Set 3, per country. SWEDEN USA SET 3 HF SE SP CA VP DC MP SET 3 HF SE SP CA VP DC MP Hndsfree Hndsfree Selfview Selfview Still Picture Still Picture Cmer Cmer Videophone Videophone Doc. Cmer Doc. Cmer 9 - Microphone 8 - Microphone 13 - Missing 2 2 Missing 7

162 TABLE 5. Men semntic scle rtings of the 21 grphicl symbols, per country (SWE - Sweden, n=48, USA, n=40) Meningful Concrete Fmilir Simple Cler SW E US A SW E US A SW E US A SW E US A SW E US A [1] Cmer [2] Cmer [3] Cmer [4] Doc. Cmer [5] Doc. Cmer [6] Doc. Cmer [7] Hndsfree [8] Hndsfree [9] Hndsfree [10] Microphone [11] Microphone [12] Microphone [13] Selfview [14] Selfview [15] Selfview [16] Still Picture [17] Still Picture [18] Still Picture [19] Videophone [20] Videophone [21] Videophone Overll Mens

163 PAPER 7 ON THE MERITS OF USING MULTIPLE INDICES IN EVALUATING SMALL GRAPHICAL SYMBOLS