Visual Attention. International Lecture Serie. Nicolas P. Rougier. INRIA National Institute for Research in Computer Science and Control

Transcription

1 Visual Attention International Lecture Serie Nicolas P. Rougier ( ) INRIA National Institute for Research in Computer Science and Control National Institute of Informatics Tokyo, December 2, / 37

2 Outlook.1. We see only what we look at.2. What is Attention?.3. Visual Attention.4. Theories of Visual Attention.5. Behavioral perspectives 2 / 37

3 We see only what we look at (Maurice Merleau Ponty, 1961) 3 / 37

4 Slow change blindness (O'Regan, 2001) ( From J.Kevin O Regan ( 4 / 37

5 Slow change blindness (O'Regan, 2001) First frame Last frame 5 / 37

6 How blind are you? ( Transport for London campaign to make drivers aware how easy it is to miss cyclists on the road and make cyclists understand how difficult they are to see. 6 / 37

7 Eyes and retina Images on retina are formed upside-down There is a blind spot on the retina where optic nerves passes throught it Retina receptors are not uniformly distributed over the surface of the retina Eye is always moving even when fixing a point (micro-tremors) Coprygith (c) 2007 Ignacio Icke 7 / 37

8 The case of Stephen Wilshire (Sacks, 1995) They were images and showed us some of the immensely complex neutral processes that are needed to make a visual and graphic image. (Sacks, 1995) Copyright (c) 2005 Stephen Wiltshire 8 / 37

9 What is Attention? 9 / 37

10 What is Attention? Everyone knows what attention is. It is the possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence. It implies withdrawal from some things in order to deal effectively with others, and is a condition which has a real opposite in the confused, dazed, scatterbrained state which in French is called distraction, and Zerstreutheit in German. W. James, / 37

11 Everyday attention effects Cocktail party effect Divided auditory attention allows you to listen to a conversation while mostly ignoring others. Pop-out effect Bottom-up visual processing direct your attention to salient stimuli (loud sound, moving/growing object in the visual field, intense heat, etc.) Blindness effect Selective attention allows you to recruit processing onto specific aspects making you virtually blind to other aspects. 11 / 37

12 Early experiments Problem of air traffic controllers: hearing intermixed voices of pilots made the task quite difficult. Cherry conducted experiments where people have to separate sentences presented to each ear. Dichotic listening experiment (Cherry, 1953) Left ear: sentence A Right ear: sentence B Subjects able to report one sentence and alsmost nothing about the other. early selection theory (Broadbent, 1958) Mixed dichotic listening experiments (Gray & Wedderburn, 1960) Left ear: cat-4-mouse Right ear: 3-eats-5 Subjects report cat eats mouse and late selection theory (Deutsch & Deutsch, 1963) 12 / 37

13 Description Clinical Description (Sohlberg & Mateer, 1989) Focused To respond discretely to a specific stimuli. Sustained To maintain a consistent behavioral response Selective To maintain attention in the face of distractors Alternating To shift focus of attention Divided To respond simultaneously to multiple tasks Cognitive Description Motor movements preparation, priming, etc. Sensory auditory, visual, proprioception, etc. Overt motor response (explicit) Covert cognitive response (implicit) Top-down goal driven, bias, etc. Bottom-Up stimulus driven, pop-out, etc. 13 / 37

14 Visual Attention 14 / 37

15 Cortical connectivity (Felleman and Van Essens, 1991) Model of cortical connectivity 32 cortical areas 10 hierarchical levels Copyright (c) 1991 Felleman and Van Essens 15 / 37

16 Main visual pathways The dorsal pathway V1 V2 MT posterior parietal cortex Where or How pathway Motion and representation of object locations The ventral pathway V1 V2 V4 inferior temporal cortex What pathway Form and object representation The frontal pathway Executive control Temporal organization of behavior Visual Awareness 16 / 37

17 Visual pathways (Itti & Koch, 2001) Visual information Visual cortex Inferotemporal cortex Posterior parietal cortex Prefrontal cortex Motor and other systems Superior colliculus Memory and cognition Eye movement 17 / 37

18 Spotlight metaphor Behavioral level Attention is the capacity to select a relevant region of the sensory space Topological region of the sensory space spatial attention Featural region of the sensory space feature oriented attention Object as such object oriented attention 18 / 37

19 Exogeneous and endogeneous factors Exogeneous visual attention (Desimone & Duncan, 1995) Visual attention is driven by physical properties of stimuli color orientation movement curvature Endogeneous visual attention (Yarbus, 1967) Visual attention is biased by a priori knowledge and goals. 19 / 37

20 Faciitation and suppression Facilitation & suppression, non spatial attributes Influence of novelty in LIP Influence of relevant attributes IT: complex objects (e.g. faces) V4: simple attributes (e.g. color, orientation) MT: movements (e.g. speed, direction) Facilitation & suppression, spatial attributes Directing spatial attention to a stimulus increases its effective contrast (Reynolds et al., 1999) Inhibition of return (Posner et al., 1980; Klein, 2000) 20 / 37

21 Visual Search Where is Waldo? 21 / 37

22 Parallel Search It s pretty easy to find the X among the L s. 22 / 37

23 Parallele Search Still easy 23 / 37

24 Parallel Search Still easy 24 / 37

25 Sequential Search Much harder / 37

26 Sequential Search 26 / 37

27 Theories of Visual Attention 27 / 37

28 Theories of Visual Attention Features Integration Theory (Treismann & Gelade, 1980) Serial search Parallel search Reaction time (ms) Number of items Several primary visual features are processed and represented with separate feature maps that are later integrated into a saliency map. Parallel search Pre-attentive attention Sequential search Attentive attention 28 / 37

29 Theories of Visual Attention Automatic vs Controlled (Stroop, 1935) Say color of words out loud as quickly as possible yellow red green blue red green yellow blue red green blue yellow Reading is quite automatic and color naming requires control (suppression) of reading, thus it is slower. 29 / 37

30 Theories of Visual Attention Inhibition of return (IOR, Posner, 1980) Fixation frame Time Cue Uncued Cued 30 / 37

31 Theories of Visual Attention Inhibition Of Return (IOR, Posner, 1980) IOR operates to decrease the likelihood that a previsously inspected item in the visual scene will be reinspected (Klein, 2008) Valid for mobile targets (Tipper et al., 1991) updated via perception Up to five indices (Pylyshin, 2004) Valid only when spatial working memory is available implied memorization of previously attended targets Appears after a time dependent of task difficulty Neurons dynamic does not drive IOR 31 / 37

32 Theories of Visual Attention Saliency Maps (Itti & Koch, 2001) Saliency map is a topographically arranged map that represents visual saliency of a corresponding visual scene. (Niebur, 2007) Localization Frontal Eye Field (FEF)? LGN (Lateral Geniculate Nucleus)? Superior Colliculus? Distributed? 32 / 37

33 Theories of Visual Attention Premotor Theory of Attention (Rizzolati, 1987) No need to postulate for two distinct control mechanisms One dedicated circuit for action One dedicated circuit for attention The premotor theory of attention postulates that attention may derive from weaker activation of same frontal-parietal circuits. 33 / 37

34 Behavioral perspectives Modeling perspectives Functional separation between What and Where pathways Non spatialy guided attention to facilitate processing of attributes Spatialy guided attention to facilitate processing of a spatial location Spatial attention to be deployed sequentially IOR to avoid attending a previously attended location Computational perspectives To represent attention focus from saliency map To re-use spatal attentional model To memorize already attended locations To allow updating through perception Dynamic spatial working memory To dynamically inhibit point of attention To temporally control inhibition effect 34 / 37

35 Behavioral perspectives Perception in action (Gibson, 1979) Without perception action would be unguided Without action perception would serve no purpose Sensori-motor account on vision (O Regan & Noë, 2001) Refute the hypothesis of an internal representation of the world The outside world serves as its own, external representation To master the laws of sensorimotor contingency Deictic codes for the embodiment of cognition (Ballard et al., 1997) System of implicit reference (called deictic) to bind objects to cognitive programs External frame of reference centered at the fixation point 35 / 37

36 Conclusion On ne voit que ce qu on regarde (Merleau-Ponty, 1961) Visual perception and attention are quite different from our unified visual experience and implies a lot of different and complex processings: Parallel/Serial Attentive/Pre-attentive Automatic/Controlled Conscious/Unconscious The challenge for computational neuroscientist is thus to handle this complexity within a unified model in order to understand attention and makes the link to cognition. 36 / 37

37 Bibliography Some references Posner, M.I., Rafal, R.D., Choate, L.S. & Vaughan, J. (1985). Inhibition of return: Neural basis and function. Cognitive Neuropsychology, 2, Rizzolatti, G., Riggio, L., Dascola, I. & Umiltà C. (1987). Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention, Neuropsychologia 25: Ballard, D., Hayhoe, M., Pook K. & Rao, R. (1997). Deictic codes for th embodiment of cognition, Behavioral and Brain Science 20:4. O Regan, K. & Noë, A. (2001). A sensorimotor account of vision and visual consciousness, Behavioral and Brain Sciences 24, Itti, L. & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2, / 37