Eye Tracking and Art A Case Study of Integrating Innovations in Research in the Undergraduate Classroom Herbert W. Helm, Jr. Karl G. D. Bailey
Undergraduate research a term that encompasses scholarship and creative activity is recognized as a highimpact educational practice that has the ability to capture student interest and create enthusiasm for and engagement in an area of study. --Rowlett, Blockus, & Larson (2012) Characteristics of Excellence in Undergraduate Research
How can faculty effectively mentor undergraduate researchers? complications: available time high student course loads high teaching loads lack of prior student experience student research anxiety
faculty research + = student classroom research training
cognitive psychology faculty research + = student classroom research training eye tracking: artist control of eye movements
Faculty Research Interest: the ability of artists to guide eye movements. Student Learning Objective: the roles of top-down and bottom-up processing in perception.
Quantifying Eye Movements Eye movements are a useful index of current cognition because: 1. The size of the fovea requires the eye to be moved in order to maintain high acuity. 2. Although attention and vision can be decoupled, in practice they are closely synchronized. 3. In order to protect the contents of cognition, the eye is not moved until processing at a given location is sufficiently completed. 4. The fovea can only be pointed at a single location at a time; visual input is functionally serial (to some degree). 5. Eye movements are energetically cheap (relative to other bodily movements).
x position y position sample time pupil quality I. Select a segment of time to analyze. 1 s = 1000 ms = 60 samples II. Remove blinks. III. Remove saccades, leaving defined fixations. IV. Compare the x-y coordinates of fixations to defined regions of interest. x position y position sample start time length ROI
V. Calculate differences in attention directed to different regions of interest. proportion of time spent fixating regions of interest (ROIs) 1 s = 1000 ms = 60 samples
IV. Compare the x-y coordinates of fixations to defined regions of interest. s1 VI. Calculate average differences in attention directed to ROIs. s2 s3 s4 s5 s6 s7 s8 s9 s10 AVERAGE 1 s = 1000 ms = 60 samples
VIII. Create graphical depictions of how eye movements tend to cluster on the image. Heat Map: shows the smoothed proportion of subjects who fixated a given region of the image during the selected time segment. 1 Saccade Map: shows the typical directions and consecutive locations of eye movements. 3 2 Fixation Map: shows the location of each fixation during the selected time segment.
Can artists guide and predict viewers eye movements? consider: 1. rules of composition 2. human vision as a reality engine that generates immersive visual experiences (Hoffman, 2011; Livingstone, 2002)
street scene watercolor artist John Salminen (Duluth, MN)
landscape watercolor artist Terry Armstrong (Warsaw, IN)
Is there a top-down effect of task on eye movement patterns or do bottomup stimulus properties exert the majority of control? 44 subjects task between subjects (21 gallery/23 memory) image type within subjects (6 landscape/8 street scenes viewed for 30 seconds) analysis of initial 10 seconds of viewing 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 GALLERY MEMORY street scene (8 images by Salminen) landscape (6 images by Armstrong) Total Number of Fixations per Image per Subject LANDSCAPE STREET LANDSCAPE STREET INSTRUCTION TYPE LANDSCAPE STREET
1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions landscape (1 of 6 images by Armstrong) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions landscape (1 of 6 images by Armstrong) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions landscape (1 of 6 images by Armstrong) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions landscape (1 of 6 images by Armstrong) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions landscape (1 of 6 images by Armstrong) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions landscape (1 of 6 images by Armstrong) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
3 2.5 2 1.5 1 0.5 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
2.5 2 1.5 1 0.5 0 ROI 1 ROI 2 ROI 3 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
4 3.5 3 2.5 2 1.5 1 0.5 0 ROI 1 ROI 2 ROI 3 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
2.5 2 1.5 1 0.5 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 gallery instructions memory instructions street scene (1 of 8 images by Salminen) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
If you could show people pictures and track their eyes, what questions would you ask?
Course Organization introduction to research literature cognitive psychology content sample experiments experiment design (within constraints) & data analysis product: abstract written to meet MPA Psi Chi session criteria
art & vision thought & language multitasking with cell phones expertise perception selective attention long-term memory language knowledge long-term memory divided attention working memory imagery everyday memory problem solving reasoning 2 x 2 design testing interaction
art & vision DeAngelus & Pelz (2009) Vogt & Magnussen (2007) Helm & Bailey original study perception selective attention long-term memory class-selected between-subjects variable X lab group-selected images and regions of interest We have discussed the effects of task and expertise on viewing art. We will decide on a task manipulation or personality factor as a between subject variable in class. Your group must decide on a stimulus variable to manipulate within subjects. This means that you will need to collect or create at least 1 stimulus for each level of your stimulus variable (e.g. color watercolor painting of fish vs. black and white version of painting) and counterbalance those stimuli. All stimuli will be collected and pseudorandomized; each class member will be responsible for bringing one subject to the lab and helping to run the eyetracker while data is collected for all groups.
interactive eye movement analysis program
Assessment How well do students respond to integrated research in the classroom? Student research attitudes and self-efficacy: Enjoyment; Research Anxiety; Perceived Usefulness; Skill Self-Efficacy (Bailey, in preparation; based on Papanastasiou, 2005, and Kardash, 2000) Need for Cognition (Cacioppo, Petty, & Kao, 1984) Grit (Duckworth & Quinn, 2009; Duckworth, Peterson, Matthews, & Kelly, 2007) Academic Self-Regulation (Vansteenkiste, Sierens, Soenens, Luyckx, & Lens, 2009) Student Perception of Learning Student Performance Peer-Review of Lab Groups
Assessment Students felt that they learned both content and skills, but were more confident that they had learned about designing and running experiments. Students remained very uncomfortable with organizing and formatting data for analysis. Gains in skills self-efficacy were correlated with Grit (r = 0.424, p < 0.05). Reduction in research anxiety was negatively correlated with Need for Cognition (r = 0.435, p < 0.05). Students improved in their ability to recognize interactions and main effects, but still had some difficulty in identifying variables. No significant changes occurred in enjoyment, skills, anxiety, or perceived usefulness of research.
What We Know So Far Students are interested in learning through research; however, not all students have the necessary grit or motivation to persist through a semester of class-related research. Integrating faculty research into the classroom helps to clarify the methods, basic assumptions, and analyses involved in that research, especially if it is in progress. Teaching students how to work with raw data is one of the most difficult tasks facing both students and teachers when integrating research. Eye tracking seems to genuinely interest students; solving interesting research problems pushed students to think critically about data.
Acknowledgements: Research Assistants: Jeremy Grant Elkyn Beltré Olivia Titus
1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 ROI 1 ROI 2 ROI 3 ROI 4 ROI 5 ROI 6 ROI 7 ROI 8 gallery instructions memory instructions street scene (1 of 2 images by Helm) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%
3 2.5 2 1.5 1 0.5 0 ROI 1 ROI 2 ROI 3 ROI 4 ROI 5 ROI 6 gallery instructions memory instructions object scene (1 of 2 images by Helm) Average Number of Fixations / Subject (in 10 seconds) GALLERY MEMORY > 75% subjects fixating area 0%