The mind-writing pupil

Transcription

1 The mind-writing pupil Near-perfect decoding of visual attention with pupillometry Sebastiaan Mathôt Jean-Baptiste Melmi Lotje van der Linden Stefan Van der Stigchel The research leading to these results has received funding from the People Programme (Marie Curie Actions) of the European Union's Seventh Framework Programme (FP7/ ) under REA grant agreement n

2 Pupil light response The pupil constricts when you Look at brightness Attend to brightness Binda et al. (2013), Binda & Murray (2015) Mathôt et al. (2013, 2014, 2015), Naber et al. (2013) Reviewed in Mathôt & Van der Stigchel (in press)

3 Attention + Pupillometry = Human-computer interface

4 PCI A Pupil size B A B

5 PCI A B Stop when sufficiently sure! Biofeedack

6 PCI H A G F B C E D

7 PCI H B F D

8 PCI H D

9 PCI D

10 What it looks like

11 Methods Preregistered 10 untrained participants 4 phases Select target from 2, 4, and 8 options Free writing with virtual keyboard Success criterion: > 80% selection accuracy Control for eye movements Pause when fixation lost Gaze-stabilization mode

12 Pupil size

13 Two options Accuracy: 89% Selection time: 14.9 s Success: 9/10 pp

14 Four options Accuracy: 91% Selection time: 20.2 s Success: 9/9 pp

15 Eight options Accuracy: 88% Selection time: 28.0 s Success: 9/9 pp

16 Free writing

17 Free writing

18 Free writing Enfin terminée Finally done! Je m'apelle ***** My name is ***** Je vais agrandir I'm going to get bigger Je suis **** I am **** Le chat dort The cat sleeps Expérience terminée Experiment finished Vive le poiil? Live the fuur? Le chien boit The dog drinks Je ne suis pas si rapide que ça I'm not all that fast

19 Conclusions Decoding attention through pupillometry All participants: Met 70% criterion for brain-computer-interface (BCI) use (e.g. Birbaumer, 2006) 9/10 participants: Met our 80% criterion Comparable to decoding of single-unit electrodes (Astrand et al., 2014) Comparable to the best covert-attention (EEG) BCIs (e.g. Treder et al., 2011) but more practical Software will be made freely available Based on OpenSesame, PyGaze, and PsychoPy (Mathôt, Schreij, & Theeuwes, 2012; Dalmaijer, Mathôt, Van der Stigchel, 2014; Peirce, 2007) Poster last Tuesday!

20 Applications Communication with locked-in patients Ultra-secure password input Training and assessment of sustained attention

21 Enfin terminé The mind-writing pupil Thank you for your attention Sebastiaan Mathôt Mathôt & Van der Stichel (in press). New light on the mind's eye: The pupillary light response as active vision. Current Directions in Psychological Science. The research leading to these results has received funding from the People Programme (Marie Curie Actions) of the European Union's Seventh Framework Programme (FP7/ ) under REA grant agreement n

22 References Astrand, E., Wardak, C., & Ben Hamed, S. (2014). Selective visual attention to drive cognitive brainmachine interfaces: from concepts to neurofeedback and rehabilitation applications. Frontiers in Systems Neuroscience, 8. doi: /fnsys Naber, M., Alvarez, G. A., & Nakayama, K. (2013). Tracking the allocation of attention using human pupillary oscillations. Frontiers in Psychology, 4. doi: /fpsyg Binda, P., & Murray, S. O. (2015). Spatial attention increases the pupillary response to light changes. Journal of Vision, 15(2), 1. doi: / Binda, P., Pereverzeva, M., & Murray, S. O. (2013). Attention to bright surfaces enhances the pupillary light reflex. The Journal of Neuroscience, 33(5), doi: /jneurosci Birbaumer, N. (2006). Breaking the silence: Braincomputer interfaces (BCI) for communication and motor control. Psychophysiology, 43(6), doi: /j x Dalmaijer, E., Mathôt, S., & Van der Stigchel, S. (2014). PyGaze: An open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments. Behavior Research Methods, 46(4), doi: /s Mathôt, S., & Van der Stigchel, S. (in press). New light on the mind's eye: The pupillary light response as active vision. Current Directions in Psychological Science. Mathôt, S., van der Linden, L., Grainger, J., & Vitu, F. (2013). The pupillary response to light reflects the focus of covert visual attention. PLoS ONE, 8(10), e doi: /journal.pone Mathôt, S., Dalmaijer, E., Grainger, J., & Van der Stigchel, S. (2014). The pupillary light response reflects exogenous attention and inhibition of return. Journal of Vision, 14(14), 7. doi: / Mathôt, S., van der Linden, L., Grainger, J., & Vitu, F. (2015). The pupillary light response reflects eye-movement preparation. Journal of Experimental Psychology: Human Perception and Performance, 41(1), doi: /a Mathôt, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 44(2), doi: /s Treder, M. S., Schmidt, N. M., & Blankertz, B. (2011). Gaze-independent braincomputer interfaces based on covert attention and feature attention. Journal of Neural Engineering, 8, e doi: / /8/6/066003