EXPLORING COGNITIVE STRATEGIES FOR INTEGRATING MULTIPLE-VIEW VISUALIZATIONS

Transcription

1 EXPLORING COGNITIVE STRATEGIES FOR INTEGRATING MULTIPLE-VIEW VISUALIZATIONS Young Sam Ryu 1, Beth Yot 2, Gregorio Convertino 2, Jian Chen 2, and Chri North 2 Grado Department of Indutrial and Sytem Engineering 1 Department of Computer Science 2 Virginia Polytechnic Intitute and State Univerity Blackburg, Virginia Viualizing information i ueful for finding pattern in complex data et, but little reearch ha been done on how people undertand multiple-view viualization (multiple viualization preented imultaneouly). A controlled experiment wa performed uing different combination of viualization and different tak type a independent variable, and qualitative and quantitative data were collected. To collect the data pychological tet, log of the participant interaction, eye-tracking equipment, and video recording were ued. Thi paper report a portion of the reult from thi experiment. Main finding include that, contrary to what wa uggeted in previou literature, the time cot for witching between different type of viualization (context witching) may not be ignificant, and that diplaying the data uing the ame type of viualization may caue interference. Orthogonal combination appear to aid uer in finding and recognizing pattern, and focuing attention and analogical reaoning on patial relationhip may be important cognitive abilitie for the given tak. INTRODUCTION Viualizing information can provide inight into complex data et. Without viualization, uing only tatitic and table of information, pattern would be harder to detect. Variou companie and government agencie make ue of information viualization for purpoe ranging from credit card ecurity to exploring the human genome to creening for terrorit. We hope that thi tudy, in combination with follow-up reearch, will help in the creation of criteria to improve the deign of thee application. When viualizing information it can be preented in a ingle integrated view or in multiple coordinated view. The reearch literature preent two oppoite line of thought regarding multiple-view veru integrated-view viualization. On one hand, ome author emphaize the advantage of layering information and integrating more information in one view (Burn, 2000; Tufte, 1983). On the other ide, different author argue for the advantage of plitting information into multiple-view (Robert, 1998). If multiple-view are ued, it i poible to preent them imultaneouly or equentially. Burn tudied what we refer to a both multiple-view and integrated-view viualization preented ynchronouly and aynchronouly (he categorize thee dimenion a low-pace or high-pace and low-time or high-time) (Burn, 2000). Related to what Burn would claify a low-time high-pace, Trafton et al. tudied multiple viualization preented to Navy meteorologit equentially (Trafton, Kirchenbaum, Tu, Miyamoto, Balla, & Raymond, 2000). In thi paper we focu on multiple-view viualization preented imultaneouly (or high-time low-pace). The multiple-view viualization we ued conited of two viualization connected in uch a way that electing data in one view would highlight related data in the other view. Figure 1 how an example of a multiple-view ytem with coordinated view. Figure 1. Screenhot of SNAP, a ytem howing multipleview viualization ( The intention of thi reearch i to tudy the way in which people mentally integrate data from multiple-view viualization by invetigating how uer identify the relationhip between data in different view. The time cot of context witching i meaured and the cognitive procee involved in multiple-view viualization are explored. The motivation for doing thi reearch i at leat three-fold. Firt, ome guideline (Baldonado, Woodruff, & Kuchinky, 2000) for deciding when to ue multiple-view are available, but urpriingly little ha been done to confirm thee. Second, there ha been a growing interet in uing multi-view viualization (North & Shneiderman, 2000; Suh, Woodruff, Roenholtz, & Gla, 2002) and of undertanding cognitive procee of uer. Third, deigning effective multiple view ytem i a challenging tak. Guideline for uing multi-view ytem ugget their ue if the data contain divere attribute, if correlation and/or diparitie in data can be made apparent, or if a ingle view i

2 overwhelming. A number of cognitive apect are involved with their ue including the time and effort to learn the ytem, the load on the uer working memory, the effort required for comparion, and context witching (Baldonado et al., 2000). The cot of context witching between view i cited multiple time in thee guideline and i an important concern ince a uer mut make ue of numerou perceptual cue, and cognitive load may be increaed. In thi reearch, we are intereted in meauring the cognitive apect involved in multi-view ytem. Baldonado et al. (2000) alo pointed out that beide occupying more pace and requiring the learning of additional contraint, multiple view ituation increae the demand on cognitive attention. Thi ugget that attention i a cognitive function that will dratically influence the uer performance. More pecifically, the tak will motly involve the ue of elective attention, where the uer eek information and earche for target. Wicken and Holland (2000) tated that although elective attention can occur without a change in direction of gaze, mot of the time it i till the cae that, our gaze i driven by our need to attend. Studying viual canning behaviour, which i cloely related to the concept of an attentional earchlight, can reveal inight into elective attention (Rock & Mack, 1994). For thi reaon, eye-tracking wa ued in thi reearch. It i important to note that the reult preented in thi paper are part of a larger work. The focu of another paper (Convertino, Chen, Yot, Ryu, & North, 2003) i the eyetracking data and the logging of uer interaction with the ytem. Here we preent ome of the baic finding from thoe apect of the experiment, but alo delve into the data obtained from the videotape and from uer comment. METHOD Deign The goal of thi experiment wa to gain inight into the cognitive trategie people ue to mentally integrate multipleview viualization. We ued a 4x4 factorial, within-ubject deign with participant. The order of condition for each ubject wa balanced by Latin-quare deign. Independent variable were the combination of viualization and quetion type that varied in difficulty. The viualization ytem ued in thi tudy wa implemented in OpenGL. Thi ytem upported the traditional interaction technique (the ability to elect a ingle point or to drag and elect multiple point), and preented the uer with pair of viualization diplaying different attribute of the ame data et. Uing the bruhing and linking interaction technique, if a point wa elected in one view, the correponding point wa alo highlighted in the other view. The given combination were two parallel coordinate plot (PP), a parallel coordinate plot and a catter plot (PS), a parallel coordinate plot and a geographic map (PG), and a catter plot and map (SG) (Figure 2). All the ubject had been expoed to all four type of viualization during their previou cla work. The ytem wa able to record the hitory of the uer action and the correponding completion time. The Ican RK-464 eye-tracker wa ued to track ubject eye movement. Figure 2. Combination from top left clockwie: PP, PS, SG, PG. At the bottom left of each are the pattern, and the bottom right the quetion. Table 1. Experimental deign ummary (within-ubject ) Quetion 1 (Single witch & Search) Quetion 2 (Multiple witch & Search) Quetion 3 (Single witch & Pattern) Quetion 4 (Multiple witch & Pattern) PP PS PG SG. The data conited of 40 data point with 4 attribute (A, B, C, and D). The data wa abtract data in order to avoid the interference of previou experience. The A and B attribute were hown in the viualization in the left window and C and D in the right window. The ubject wa requeted to identify relationhip in the data. The relationhip were alway between 2 attribute, and involved 10 data point with 1 outlier. The PP condition repreented the ituation without any context witching, wherea the other three condition required context witching becaue thoe involved pair of different view (PS, SG, and PG). Within each of the four combination, four different type of quetion were given to differentiate tak difficulty. Quetion 1 required earching for a given attribute, quetion 2 required earching for and finding the lower attribute when comparing two object. Quetion 3 required finding a pattern between C and D value, and quetion 4 required finding a pattern between B and C value. Table 1 how the deign chema of the 4x4 factorial withinubject deign..

3 Procedure Sixteen ubject participated in two eion. Firt, they provided demographic including their familiarity with different viualization. They were given the Raven tet for meauring the ability to form perceptual relation and to reaon by analogy and three ubtet of the Wechler Adult Intelligence Scale Revied (WAIS-R) including Digit Symbol, Picture Completion, and Digit Span Verbal ubtet. Each of the ubtet give an index about ome pecific cognitive function. The Digit Symbol performance ubtet involve viual-motor coordination and peed. The Picture Completion performance ubtet involve viual recognition, general information, and focuing attention. Finally, the Digit Span verbal ubtet involve auditory attention, concentration, and hort-term memory. During the econd eion participant were given intruction, completed two practice tak, and then performed experimental tak. During the practice tak the ubject could familiarize themelve with the ytem and the different viualization. They were videotaped and aked to think aloud. After completing thee tak, they were aked to rate the difficulty on Likert cale of each quetion type and of uing the four pair of viualization. Finally, they ranked their preference of combination and quetion. RESULTS Uer Performance The completion time for each quetion wa divided by the average completion time to account for individual pace. Figure 3 how completion time per quetion type. Analyi of variance (ANOVA) wa conducted and the reult howed that PP took ignificantly longer than the other, F (3, 45)=11.32, p< Time (%) Quetion Type v. Completion Time Tak PP PS PG SG Figure 3. Completion time weighted by an individual pace. Subject anwered the mot quetion incorrectly with PP - 28 wrong anwer out of 64 quetion ( ubject x 4 quetion), and ubjectively rated PP a the mot difficult combination to ue, but the difference wa not ignificant uing an ANOVA, F(3, 45)=2.08, p= PS wa the bet combination with the mot correct anwer a 14 wrong anwer out of 64 quetion (Table 2) and lowet completion time a hown in Figure 3. Thee reult were oppoite the hypothei and literature ince PP took longer than thoe preenting data in different manner. Familiarity with parallel coordinate plot wa not correlated with completion time (r=- 0.13). The difference in completion time may be a reult of interference from the ame viualization, or from lack of orthogonal repreentation. Table 2. Wrong anwer and average difficulty rating Total number of wrong anwer from 64 quetion Average difficulty rating (1=leat difficult, 5=mot difficult) PP PS PG SG Subjective Rating There wa no ignificant difference between preference rank and difficulty rating baed on the combination of viualization. However, ubject did prefer and therefore rank the combination according to their perceived level of difficulty (Table 2). Thi pattern did not hold for quetion type. The difficulty rating wa the lowet for quetion type 1, and the highet for quetion type 4 a intended. They alo preferred the quetion involving a earching tak. Additionally, ubject preferred the quetion that required multiple viual witche between view to thoe that required a ingle viual witch. Uer were aked to comment on their opinion of the combination of viualization on the final quetionnaire. Their repone confirmed that they found parallel coordinate the mot difficult viualization to ue. Five of the ubject commented that the map wa the eaiet type of viualization to ue, while two commented that it wa harder. Four of the ubject commented that relationhip were harder to dicover uing parallel coordinate and that in general parallel coordinate were harder to ue. Three ubject felt the oppoite about parallel coordinate, preferring them to the other type of viualization and ranking them the eaiet to ue. One uability iue when uing parallel coordinate i that becaue the line cro, it can be hard to elect multiple edge. Another iue three ubject mentioned wa the difficulty in keeping their head till for the duration of the experiment (a neceity becaue of the eye-tracking equipment). From the think-aloud data, it appear that when uer were anwering quetion type 4 and had to elect a range of A value, and find a relationhip between B and C value, much of the uer' focu wa directed toward elected the range of A value. Some uer commented that finding pattern between attribute preented in different view wa very challenging, and one aid he did not know how to do it. Technical difficultie and time contraint have temporarily prevented more in-depth analyi of thi data. However, the video

4 recording alo captured the creen and action of uer. Viewing thee tape will help dicover how uer took advantage of multiple-election and when they choe to ue ingle-election over multiple-election along with more detail regarding their cognitive trategie. It hould alo provide inight into interaction trategie characterizing ucceful uer. Cognitive Abilitie From the WAIS-R given to each ubject, the picture completion ubtet core howed relatively high correlation with completion time (r=-0.46) (Figure 4) and the number of error (r=-0.4). The ubject performance on the Raven tet wa correlated negatively with the number of error on quetion 3 (r=-0.44) and the correlation increaed in trength if quetion 1 and 3 were conidered together (r=-0.52). The Raven tet meaure reaoning by analogy uing a et of viual timuli. During the tet, each ubject had to detect a relationhip and then reapplied the ame relationhip to other timuli by analogy. Average Tak Completion Time v. WAIS Picture Completion Score 120 Average 100 Completion 80 Time (ec) WAIS Picture Completion Score Figure 4. Higher Picture Completion core correlated with lower completion time. Eye-tracking Data An eye-fixation i each intance that the eye remained in an area that wa 5 pixel vertically and 3 horizontally for at leat 40 milliecond. The majority of fixation time within the right parallel coordinate plot wa pent in between the axe. Thi ugget that uer were looking at the lope of the line and not at the value on the axe to determine pattern. It alo appear that uer that performed wore fixated on one location for longer period of time. For more detailed eyetracking reult pleae refer to Convertino et al. (2003). DISCUSSION Uer Performance and Subject Rating We had predicted that ituation involving two parallel coordinate plot (PP) would take the leat amount of time becaue there would be no cognitive load from context witching. The reult were contrary to our hypothei becaue quetion involving the ue of two parallel coordinate plot (PP) actually took ignificantly longer time to complete. Thi ugget that in ome ituation, cognitive integration may actually be more difficult when a peron i preented with two identical viualization. Another explanation i that uer were able to match pattern eaier when preented in a catter plot ince the choice were preented in an orthogonal manner. Reult from the uer comment on the final interview and from the think-aloud data ugget finding pattern uing parallel coordinate could be challenging, and that finding pattern acro viualization wa hard. It hould be noted that the trength of uing parallel coordinate i that multiple attribute can be plotted in the ame viualization and their ue wa intentionally implified for experimental purpoe. Cognitive Abilitie and Eye-Tracking Data The relatively trong correlation between the picture completion ubtet and tak completion time ugget that the tak required conitent involvement of viual recognition and focuing attention. In addition, the ubject performance on the Raven tet wa correlated negatively with the number of error on quetion type 1 and 3. Since the Raven tet meaure reaoning by analogy uing a et of viual timuli, thi ugget that analogical reaoning i required to recognize relationhip between two attribute in a ingle viualization, but i not a important when finding pattern acro viualization. Reult from the eye-tracker how that when the parallel coordinate plot wa preented on the left the ubject pent mot of their time looking at the axe. Thi wa to be expected ince mot quetion aked the ubject to ue a ubet of point from one of thee axe. More intereting wa the ituation of PP. In thi condition, ubject had to find pattern within the parallel coordinate plot on the right. Subject actually pent mot of their time looking at the area in between the two axe. Thi ugget that mot of the ubject were looking at the lope of the line, and not the value on the axe in order to find the pattern. For more detail and dicuion on the reult of the eye-tracking, along with an additional dicuion of 1-dimenional veru 2-dimenional election when uing different combination of viualization, pleae ee (Convertino et al., 2003). Abtract Data Vere Domain Specific Data Burn reearch ha hown that high-time and high-pace (or integrated view viualization) upport performance better than multiple-view viualization preented either imultaneouly or equentially (Burn, 2000). She argue that the reaon for thi i that integrated-view viualization are able to how the data in a meaningful way. That particular piece of reearch applied to a certain domain, pecifically, to power plant operator. For that cenario, howing the data in a meaningful way could be baed on domain knowledge. In contrapoition with that logic, the explicit purpoe of our tudy wa to avoid the influence of factor related to pecific domain and focu on more general information

5 viualization iue. The reult of that deciion i that our reult do not apply pecifically to any domain. However, they are important iue that hould be conidered during the proce of deigning a ytem for any domain, and particularly for ituation where multiple-view are ued for exploring unknown relationhip in the data. Deigner of multiple-view ytem hould conider the poible affect of uing orthogonal view, and the poibility that interference may occur if imilar viualization are ued. CONCLUSION overview+detail document interface. In Proceeding of CHI'2002, Minneota, Trafton, J. G., Kirchenbaum, S. S., Tu, T. L., Miyamoto, R. T., Balla, J. A., & Raymond, P. D. (2000). Turning picture into number: Extracting and generating information from complex viualization. International Journal of Human-Computer Studie, 53(5), Tufte, E. R. (1983). Viual diplay of quantitative information. Chehire, CT: Graphic Pre. Wicken, C. D., & Holland, J. (2000). Engineering pychology and human performance (3rd ed.). Upper Saddle River, New Jerey: Prentice Hall. We have explored the cognitive abilitie involved in working with multiple-view viualization and the effect of context witching. Uing cognitive ability pretet we were able to find correlation between focuing attention, analogical reaoning, and performance. Additionally, our tudy ha hown that context witching may not increae the difficulty of cognitive integration. Similar viualization may caue interference reulting in decreaed performance. An alternate explanation i that ubject had to mentally tranform pattern in parallel coordinate plot to an orthogonal repreentation and thi additional tep reduced performance. Future work include evaluating different combination of viualization not explored in thi reearch. Eventually, following effort in thi direction could reult in developing a tructured method for deigning multiple-view viualization ytem baed on cognitive abilitie, tak, and dataet. For now, we recommend that deigner of multiple-view ytem conider uing orthogonal combination of viualization, and that they are aware that cognitive interference may potentially occur if imilar viualization are preented imultaneouly. REFERENCES Baldonado, M. Q., Woodruff, A., & Kuchinky, A. (2000). Guideline for uing multiple view in information viualization. In Proceeding of AVI2000, Palermo, Italy. Burn, C. M. (2000). Putting it all together: Improving diplay integration in ecological diplay. Human Factor, 42(2), Convertino, G., Chen, J., Yot, B., Ryu, Y., & North, C. (2003). Exploring context witching and cognition in dualview coordinated viualization. To appear in proceeding of cmv 2003, london. North, C., & Shneiderman, B. (2000). Snap-together viualization: A uer interface for coordinating viualization via relational chemata. In Proceeding of AVI2000, Palermo, Italy. Robert, J. C. (1998). On encouraging multiple view for viualization. In Proceeding of IV'98, London, Rock, I., & Mack, A. (1994). Attention and perceptual organization. In S. Balletero (Ed.), Cognitive approache to human perception (pp ). Hilldale, NJ: Lawrence Earlbaum Aociate. Suh, B., Woodruff, A., Roenholtz, R., & Gla, A. (2002). Popout prim: Adding perceptual principle to