Bottom-up and top-down processing in visual perception
|
|
- August Long
- 6 years ago
- Views:
Transcription
1 Bottom-up and top-down processing in visual perception Thomas Serre Brown University Department of Cognitive & Linguistic Sciences Brown Institute for Brain Science Center for Vision Research
2 The what problem monkey electrophysiology
3 Willmore Prenger & Gallant 10 The what problem monkey electrophysiology
4 b Yamane Carlson Bowman Wang & Connor 08 Willmore Prenger & Gallant 10 The what problem monkey electrophysiology
5 b Yamane Carlson Bowman Wang & Connor 08 Hung Kreiman et al 05 Willmore Prenger & Gallant 10 The what problem monkey electrophysiology
6 The what problem human psychophysics Thorpe et al 96; VanRullen & Thorpe 01; Fei-Fei et al 02 05; Evans & Treisman 05; Serre Oliva & Poggio 07
7 Vision as knowing what is where Aristotle; Marr 82
8 What and where cortical pathways Ungerleider & Mishkin 84
9 ventral what stream What and where cortical pathways Ungerleider & Mishkin 84
10 dorsal where stream ventral what stream What and where cortical pathways Ungerleider & Mishkin 84
11 How does the visual system combine information about the identity and location of objects?? dorsal where stream ventral what stream What and where cortical pathways Ungerleider & Mishkin 84
12 How does the visual system combine information about the identity and location of objects? Central thesis: visual attention (see also Van Der Velde and De Kamps 01; Deco and Rolls 04)? dorsal where stream ventral what stream What and where cortical pathways Ungerleider & Mishkin 84
13 Part I: A Bayesian theory of attention that solves the what is where problem Predictions for neurophysiology and human eye movement data Sharat Chikkerur Cheston Tan Tomaso Poggio Part II: Readout of IT population activity Attention eliminates clutter Ying Zhang Ethan Meyers Narcisse Bichot Tomaso Poggio Bob Desimone
14 Part I: A Bayesian theory of attention that solves the what is where problem Predictions for neurophysiology and human eye movement data Sharat Chikkerur Cheston Tan Tomaso Poggio Part II: Readout of IT population activity Attention eliminates clutter Ying Zhang Ethan Meyers Narcisse Bichot Tomaso Poggio Bob Desimone
15 S visual scene description I image measurements Perception as Bayesian inference Mumford 92; Knill & Richards 96; Dayan & Zemel 99; Rao 02 04; Kersten & Yuille 03; Kersten et al 04; Lee & Mumford 03; Dean 05; George & Hawkins 05 09; Hinton 07; Epshtein et al 08; Murray & Kreutz-Delgado 07
16 P (S I) P (I S)P (S) S visual scene description I image measurements Perception as Bayesian inference Mumford 92; Knill & Richards 96; Dayan & Zemel 99; Rao 02 04; Kersten & Yuille 03; Kersten et al 04; Lee & Mumford 03; Dean 05; George & Hawkins 05 09; Hinton 07; Epshtein et al 08; Murray & Kreutz-Delgado 07
17 P (S I) P (I S)P (S) S visual scene description I image measurements Perception as Bayesian inference Mumford 92; Knill & Richards 96; Dayan & Zemel 99; Rao 02 04; Kersten & Yuille 03; Kersten et al 04; Lee & Mumford 03; Dean 05; George & Hawkins 05 09; Hinton 07; Epshtein et al 08; Murray & Kreutz-Delgado 07
18 object identities object locations S = {O 1,O 2,,O n,l 1,L 2,,L n } P (S I) P (I S)P (S) S visual scene description I image measurements Perception as Bayesian inference Mumford 92; Knill & Richards 96; Dayan & Zemel 99; Rao 02 04; Kersten & Yuille 03; Kersten et al 04; Lee & Mumford 03; Dean 05; George & Hawkins 05 09; Hinton 07; Epshtein et al 08; Murray & Kreutz-Delgado 07
19 object identities object locations S = {O 1,O 2,,O n,l 1,L 2,,L n } P (S, I) =P (O 1,L 1,O 2,L 2,,L n,l n,i) S visual scene description I image measurements Perception as Bayesian inference Mumford 92; Knill & Richards 96; Dayan & Zemel 99; Rao 02 04; Kersten & Yuille 03; Kersten et al 04; Lee & Mumford 03; Dean 05; George & Hawkins 05 09; Hinton 07; Epshtein et al 08; Murray & Kreutz-Delgado 07
20 To recognize and localize objects in the scene, the visual system selects objects, one object at a time P (O, L, I) O,L visual scene description I image measurements Assumption #1: Attentional spotlight Broadbent 52 54; Treisman 60; Treisman & Gelade 80; Duncan & Desimone 95; Wolfe 97; and many others
21 Object location L and identity O are independent P (O, L, I) dorsal where stream O,L visual scene description ventral what stream I image measurements Assumption #2: what and where pathways
22 Object location L and identity O are independent P (O, L, I) =P (O)P (L)P (I L, O) dorsal where stream O object L ventral what stream location I image measurements Assumption #2: what and where pathways
23 Objects encoded by collections of universal features (of intermediate complexity) - Either present or absent - Conditionally independent given the object and its location O object L location I image measurements Assumption #3: universal features see Riesenhuber & Poggio 99; Serre et al 05 07
24 O object Objects encoded by collections of universal features (of intermediate complexity) - Either present or absent location F i position and scale tolerant features - Conditionally independent given the object and its location L X i N retinotopic features I image measurements Assumption #3: universal features see Riesenhuber & Poggio 99; Serre et al 05 07
25 O object location L F i position and scale tolerant features X i N retinotopic features I image measurements Assumption #3: universal features see Riesenhuber & Poggio 99; Serre et al 05 07
26 O object location L F i position and scale tolerant features X i N retinotopic features V2/V4 Serre et al 05 David et al 06 Cadieu et al 07 Willmore et al 10 I image measurements Assumption #3: universal features see Riesenhuber & Poggio 99; Serre et al 05 07
27 location O object Fi position and scale tolerant features Xi retinotopic features L N V2/V4 Serre et al 05 David et al 06 Cadieu et al 07 Willmore et al 10 Assumption #3: universal features I image measurements see Riesenhuber & Poggio 99; Serre et al 05 07
28 location IT Serre et al 07 O object Fi position and scale tolerant features Xi retinotopic features L N V2/V4 Serre et al 05 David et al 06 Cadieu et al 07 Willmore et al 10 Assumption #3: universal features I image measurements see Riesenhuber & Poggio 99; Serre et al 05 07
29 Predicting IT readout Serre et al 07 O object 1 IT Model Classification performance Size: Position: 3.4 o center 3.4 o center 1.7 o center 6.8 o center 3.4 o 2 o horz. 3.4 o 4 o horz. location L F i X i N position and scale tolerant features retinotopic features TRAIN TEST I image measurements Bayesian inference and attention Model: Serre et al 05 Experimental data: Hung* Kreiman*et al 05
30 O object behavior Serre Oliva & Poggio 07 location F i position and scale tolerant features L X i N retinotopic features I image measurements Bayesian inference and attention
31 O object behavior Serre Oliva & Poggio 07 location F i position and scale tolerant features L X i N retinotopic features I image measurements Bayesian inference and attention
32 O object behavior Serre Oliva & Poggio 07 location F i position and scale tolerant features L X i N retinotopic features I image measurements Bayesian inference and feedforward (bottom-up) sweep attention
33 O object behavior Serre Oliva & Poggio 07 location F i position and scale tolerant features L X i N retinotopic features I image measurements Bayesian inference and feedforward (bottom-up) sweep attention
34 O object Goal of visual perception: to estimate posterior probabilities of visual features, objects and their locations in an image location F i position and scale tolerant features Attention corresponds to conditioning on high-level latent variables representing particular features or locations (as well as on sensory input), and doing inference over the other latent variables L X i I N retinotopic features image measurements Bayesian inference and attention
35 dorsal / where O ventral / what F i L X i N I Bayesian inference and attention
36 dorsal / where O ventral / what F i L X i N P (X i I) = P (I X i ) F i,l P (Xi F i,l)p(l)p(f i ) X P (I X i ) i F i,l P (Xi F i,l)p(l)p(f i ) I Bayesian inference and attention
37 dorsal / where O ventral / what F i L feedforward input X i N P (X i I) = P (I X i ) F i,l P (Xi F i,l)p(l)p(f i ) X P (I X i ) i F i,l P (Xi F i,l)p(l)p(f i ) I Bayesian inference and feedforward (bottom-up) sweep attention
38 dorsal / where O ventral / what top-down spatial attention L F i top-down featurebased attention X i feedforward input attentional modulation N P (X i I) = P (I X i ) F i,l P (Xi F i,l)p(l)p(f i ) X P (I X i ) i F i,l P (Xi F i,l)p(l)p(f i ) I Bayesian inference and feedforward (bottom-up) sweep attention
39 dorsal / where O ventral / what top-down spatial attention L F i top-down featurebased attention X i X k feedforward input attentional modulation N P (X i I) = P (I X i ) F i,l P (Xi F i,l)p(l)p(f i ) X P (I X i ) i F i,l P (Xi F i,l)p(l)p(f i ) suppressive drive I Bayesian inference and feedforward (bottom-up) sweep attention
40 Neuron Review O The Normalization Model of Attention John H. Reynolds 1, * and David J. Heeger 2 1 Salk Institute for Biological Studies, La Jolla, CA , USA 2 Department of Psychology and Center for Neural Science, New York University, New York, NY 10003, USA *Correspondence: reynolds@salk.edu DOI /j.neuron F i R(x, θ) = A(x, θ)e(x, θ) S(x, θ)+σ L E A X i N P (X i I) = P (I X i ) F i,l P (Xi F i,l)p(l)p(f i ) X P (I X i ) i F i,l P (Xi F i,l)p(l)p(f i ) S I Bayesian inference and attention
41 P (X i = x I) F i,l P (X i = x F i,l)p (I X i )P (F i )P (L) McAdams and Maunsell 99 Model P (L = x) 1 P (L = x) =1/ L Multiplicative scaling of tuning curves by spatial attention see also Heeger & Reynolds 09
42 P (X i I) = P (I X i ) F i,l P (Xi F i,l)p(l)p(f i ) X P (I X i ) i F i,l P (Xi F i,l)p(l)p(f i ) Trujillo and Treue 02 Mc Adams and Maunsell 99 Contrast vs. response gain see also Heeger & Reynolds 09
43 P (X i = x I) F i,l P (X i = x F i,l)p (I X i )P (F i )P (L) Normalized response P stim/p cue NP stim/ P.cue P.stim/NP cue NP stim/np cue Normalized spikes Bichot et al Feature-based attention neural data from Bichot et al 05
44 O F i L X i N I Learning to localize cars and pedestrians
45 O learning object priors F i L X i N I Learning to localize cars and pedestrians
46 O learning object priors Context F i L X i N I Learning to localize cars and pedestrians
47 learning location priors (global contextual cues, see Torralba, Oliva et al) Context O learning object priors F i L X i N I Learning to localize cars and pedestrians
48 Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
49 Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
50 1st three fixations Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
51 1st three fixations Eye movements as proxy for attention ROC area Dataset: street-scenes images with cars & pedestrians and 20 without Experiment Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
52 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
53 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
54 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
55 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
56 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
57 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
58 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
59 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
60 ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment
61 Explains 92% of the inter-subject agreement! ROC area st three fixations cars pedestrians Uniform priors (bottom-up) Feature priors Feature + contextual (spatial) priors Humans Eye movements as proxy for attention Dataset: street-scenes images with cars & pedestrians and 20 without Experiment - 8 participants asked to count the number of cars/pedestrians - block design for cars and pedestrians - eye movements recorded using an infra-red eye tracker The experiment *similar (independent) results by Ehinger Hidalgo Torralba & Oliva 10
62 Method ROC area Bruce & Tsotos % Itti et al % Proposed 77.9% F i L X i N I Bottom-up saliency and free-viewing human eye data from Bruce & Tsotsos
63 Summary I
64 Summary I Goal of vision: - To solve the problem of what is where problem
65 Summary I Goal of vision: - To solve the problem of what is where problem Key assumptions: - Attentional spotlight recognition done sequentially, one object at a time - What and where independent pathways
66 Summary I Goal of vision: - To solve the problem of what is where problem Key assumptions: - Attentional spotlight recognition done sequentially, one object at a time - What and where independent pathways Attention as the inference process implemented by the interaction between ventral and dorsal areas - Integrates bottom-up and top-down (feature-based and context-based) attentional mechanisms - Seems consistent with known physiology
67 Summary I Goal of vision: - To solve the problem of what is where problem Key assumptions: - Attentional spotlight recognition done sequentially, one object at a time - What and where independent pathways Attention as the inference process implemented by the interaction between ventral and dorsal areas - Integrates bottom-up and top-down (feature-based and context-based) attentional mechanisms - Seems consistent with known physiology Main attentional effects in the presence of clutters - Spatial attention reduces uncertainty over location and improves object recognition performance over first bottom-up (feedforward) pass
68 Part I: A Bayesian theory of attention that solves the what is where problem Predictions for neurophysiology and human eye movement data Sharat Chikkerur Cheston Tan Tomaso Poggio Part II: Readout of IT population activity Attention eliminates clutter Ying Zhang Ethan Meyers Narcisse Bichot Tomaso Poggio Bob Desimone
69 The readout approach
70 The readout approach
71 The readout approach neuron 1 neuron 2 neuron 3 neuron n
72 The readout approach neuron 1 neuron 2 neuron 3 neuron n Pattern Classifier
73 The readout approach neuron 1 neuron 2 neuron 3 neuron n Prediction Pattern Classifier
74 The readout approach neuron 1 neuron 2 neuron 3 neuron n Correct Prediction Pattern Classifier
75 IT readout improves with attention train readout classifier on isolated object +
76 IT readout improves with attention train readout classifier on isolated object test generalization in clutter + +
77 IT readout improves with attention train readout classifier on isolated object test generalization in clutter unattended + +
78 IT readout improves with attention train readout classifier on isolated object test generalization in clutter unattended + + attended
79 IT readout improves with attention train readout classifier on isolated object test generalization in clutter unattended + + attended attend away
80 IT readout improves with attention Zhang Meyers Bichot Serre Poggio Desimone unpublished data
81 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
82 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
83 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
84 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
85 IT readout improves with attention Zhang Meyers Bichot Serre Poggio Desimone unpublished data
86 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
87 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
88 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
89 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
90 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
91 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
92 IT readout improves with attention + Zhang Meyers Bichot Serre Poggio Desimone unpublished data
93 Robert Desimone (MIT) Christof Koch (CalTech) Laurent Itti (USC) Tomaso Poggio (MIT) David Sheinberg (Brown)
94 Thank you!
A Bayesian inference theory of attention: neuroscience and algorithms Sharat Chikkerur, Thomas Serre, and Tomaso Poggio
Computer Science and Artificial Intelligence Laboratory Technical Report MIT-CSAIL-TR-29-47 CBCL-28 October 3, 29 A Bayesian inference theory of attention: neuroscience and algorithms Sharat Chikkerur,
More informationModels of Attention. Models of Attention
Models of Models of predictive: can we predict eye movements (bottom up attention)? [L. Itti and coll] pop out and saliency? [Z. Li] Readings: Maunsell & Cook, the role of attention in visual processing,
More information9.S913 SYLLABUS January Understanding Visual Attention through Computation
9.S913 SYLLABUS January 2014 Understanding Visual Attention through Computation Dept. of Brain and Cognitive Sciences Massachusetts Institute of Technology Title: Instructor: Years: Level: Prerequisites:
More informationModeling the Deployment of Spatial Attention
17 Chapter 3 Modeling the Deployment of Spatial Attention 3.1 Introduction When looking at a complex scene, our visual system is confronted with a large amount of visual information that needs to be broken
More informationFrom feedforward vision to natural vision: The impact of free viewing and clutter on monkey inferior temporal object representations
From feedforward vision to natural vision: The impact of free viewing and clutter on monkey inferior temporal object representations James DiCarlo The McGovern Institute for Brain Research Department of
More informationBayesian inference and attentional modulation in the visual cortex
COGNITIVE NEUROSCIENCE AND NEUROPSYCHOLOGY Bayesian inference and attentional modulation in the visual cortex Rajesh P.N. Rao Department of Computer Science and Engineering, University of Washington, Seattle,
More informationAttention March 4, 2009
Attention March 4, 29 John Maunsell Mitchell, JF, Sundberg, KA, Reynolds, JH (27) Differential attention-dependent response modulation across cell classes in macaque visual area V4. Neuron, 55:131 141.
More informationBiologically Motivated Local Contextual Modulation Improves Low-Level Visual Feature Representations
Biologically Motivated Local Contextual Modulation Improves Low-Level Visual Feature Representations Xun Shi,NeilD.B.Bruce, and John K. Tsotsos Department of Computer Science & Engineering, and Centre
More informationHow does the ventral pathway contribute to spatial attention and the planning of eye movements?
R. P. Würtz and M. Lappe (Eds.) Dynamic Perception, Infix Verlag, St. Augustin, 83-88, 2002. How does the ventral pathway contribute to spatial attention and the planning of eye movements? Fred H. Hamker
More informationUsing population decoding to understand neural content and coding
Using population decoding to understand neural content and coding 1 Motivation We have some great theory about how the brain works We run an experiment and make neural recordings We get a bunch of data
More informationObject detection in natural scenes by feedback
In: H.H. Būlthoff et al. (eds.), Biologically Motivated Computer Vision. Lecture Notes in Computer Science. Berlin, Heidelberg, New York: Springer Verlag, 398-407, 2002. c Springer-Verlag Object detection
More informationComputational Models of Visual Attention: Bottom-Up and Top-Down. By: Soheil Borhani
Computational Models of Visual Attention: Bottom-Up and Top-Down By: Soheil Borhani Neural Mechanisms for Visual Attention 1. Visual information enter the primary visual cortex via lateral geniculate nucleus
More informationEPS Mid-Career Award 2014 The control of attention in visual search: Cognitive and neural mechanisms
THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 2015 Vol. 68, No. 12, 2437 2463, http://dx.doi.org/10.1080/17470218.2015.1065283 EPS Mid-Career Award 2014 The control of attention in visual search: Cognitive
More informationA quantitative theory of immediate visual recognition
A quantitative theory of immediate visual recognition Thomas Serre, Gabriel Kreiman, Minjoon Kouh, Charles Cadieu, Ulf Knoblich and Tomaso Poggio Center for Biological and Computational Learning McGovern
More informationNetworks and Hierarchical Processing: Object Recognition in Human and Computer Vision
Networks and Hierarchical Processing: Object Recognition in Human and Computer Vision Guest&Lecture:&Marius&Cătălin&Iordan&& CS&131&8&Computer&Vision:&Foundations&and&Applications& 01&December&2014 1.
More informationKey questions about attention
Key questions about attention How does attention affect behavioral performance? Can attention affect the appearance of things? How does spatial and feature-based attention affect neuronal responses in
More informationJust One View: Invariances in Inferotemporal Cell Tuning
Just One View: Invariances in Inferotemporal Cell Tuning Maximilian Riesenhuber Tomaso Poggio Center for Biological and Computational Learning and Department of Brain and Cognitive Sciences Massachusetts
More informationAttention, short-term memory, and action selection: A unifying theory
Progress in Neurobiology 76 (2005) 236 256 www.elsevier.com/locate/pneurobio Attention, short-term memory, and action selection: A unifying theory Gustavo Deco a, Edmund T. Rolls b, * a Institució Catalana
More informationObject recognition in cortex: Neural mechanisms, and possible roles for attention
Object recognition in cortex: Neural mechanisms, and possible roles for attention Maximilian Riesenhuber Department of Neuroscience Georgetown University Medical Center Washington, DC 20007 phone: 202-687-9198
More informationATTENTIONAL MODULATION OF VISUAL PROCESSING
Annu. Rev. Neurosci. 2004. 27:611 47 doi: 10.1146/annurev.neuro.26.041002.131039 Copyright c 2004 by Annual Reviews. All rights reserved First published online as a Review in Advance on March 24, 2004
More informationintensities saliency map
Neuromorphic algorithms for computer vision and attention Florence Miau 1, Constantine Papageorgiou 2 and Laurent Itti 1 1 Department of Computer Science, University of Southern California, Los Angeles,
More informationRecurrent Refinement for Visual Saliency Estimation in Surveillance Scenarios
2012 Ninth Conference on Computer and Robot Vision Recurrent Refinement for Visual Saliency Estimation in Surveillance Scenarios Neil D. B. Bruce*, Xun Shi*, and John K. Tsotsos Department of Computer
More informationInformation-theoretic stimulus design for neurophysiology & psychophysics
Information-theoretic stimulus design for neurophysiology & psychophysics Christopher DiMattina, PhD Assistant Professor of Psychology Florida Gulf Coast University 2 Optimal experimental design Part 1
More informationCognitive Modelling Themes in Neural Computation. Tom Hartley
Cognitive Modelling Themes in Neural Computation Tom Hartley t.hartley@psychology.york.ac.uk Typical Model Neuron x i w ij x j =f(σw ij x j ) w jk x k McCulloch & Pitts (1943), Rosenblatt (1957) Net input:
More informationProbabilistic Models of the Cortex: Stat 271. Alan L. Yuille. UCLA.
Probabilistic Models of the Cortex: Stat 271. Alan L. Yuille. UCLA. Goals of the Course To give an introduction to the state of the art computational models of the mammalian visual cortex. To describe
More informationDynamic Visual Attention: Searching for coding length increments
Dynamic Visual Attention: Searching for coding length increments Xiaodi Hou 1,2 and Liqing Zhang 1 1 Department of Computer Science and Engineering, Shanghai Jiao Tong University No. 8 Dongchuan Road,
More informationA quantitative theory of immediate visual recognition
B2v8:06a=w ðdec 200Þ:c XML:ver::0: PBR V6 : 6004 Prod:Type: pp:8ðcol:fig::nilþ ED: PAGN: SCAN: Cisek, Drew & Kalaska (Eds.) Progress in Brain Research, Vol. 6 ISSN 00-62 Copyright r 200 Elsevier B.V. All
More informationThere's Waldo! A normalization model of visual search predicts single-trial human
There's Waldo! A normalization model of visual search predicts single-trial human fixations in an object search task Abbreviated title: There s Waldo! A normalization model of visual search Authors: Thomas
More informationChapter 7: First steps into inferior temporal cortex
BEWARE: These are preliminary notes. In the future, they will become part of a textbook on Visual Object Recognition. Chapter 7: First steps into inferior temporal cortex Inferior temporal cortex (ITC)
More informationNeural representation of action sequences: how far can a simple snippet-matching model take us?
Neural representation of action sequences: how far can a simple snippet-matching model take us? Cheston Tan Institute for Infocomm Research Singapore cheston@mit.edu Jedediah M. Singer Boston Children
More informationA Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex
massachusetts institute of technology computer science and artificial intelligence laboratory A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in
More informationVisual attention mediated by biased competition in extrastriate visual cortex
Visual attention mediated by biased competition in extrastriate visual cortex Robert Desimone Laboratory of Neuropsychology, NIMH, 49 Convent Drive, Building 49, Room 1B80, Bethesda, MD 20892-4415, USA
More informationREVIEWS COMPUTATIONAL MODELLING OF VISUAL ATTENTION. Laurent Itti* and Christof Koch
COMPUTATIONAL MODELLING OF VISUAL ATTENTION Laurent Itti* and Christof Koch Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up,
More informationComputer Science and Artificial Intelligence Laboratory Technical Report. July 29, 2010
Computer Science and Artificial Intelligence Laboratory Technical Report MIT-CSAIL-TR-2010-034 CBCL-289 July 29, 2010 Examining high level neural representations of cluttered scenes Ethan Meyers, Hamdy
More informationAmbiguity and invariance: two fundamental challenges for visual processing Nicole C Rust and Alan A Stocker
Available online at Ambiguity and invariance: two fundamental challenges for visual processing Nicole C Rust and Alan A Stocker The visual system is tasked with extracting stimulus content (e.g. the identity
More informationObject recognition and hierarchical computation
Object recognition and hierarchical computation Challenges in object recognition. Fukushima s Neocognitron View-based representations of objects Poggio s HMAX Forward and Feedback in visual hierarchy Hierarchical
More informationDeep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition Charles F. Cadieu, Ha Hong, Daniel L. K. Yamins, Nicolas Pinto, Diego Ardila, Ethan A. Solomon, Najib
More informationUC Merced Proceedings of the Annual Meeting of the Cognitive Science Society
UC Merced Proceedings of the Annual Meeting of the Cognitive Science Society Title Eyes Closed and Eyes Open Expectations Guide Fixations in Real-World Search Permalink https://escholarship.org/uc/item/81z9n61t
More informationA Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex
massachusetts institute of technology computer science and artificial intelligence laboratory A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in
More information10/13/2010. Brain Mechanisms in Vision Attention. Search Task. The visual system is limited in its capacity to process information,
Brain Mechanisms in Vision Attention Given The visual system is limited in its capacity to process information, Michael J. Earley, OD, PhD Professor of Clinical Optometry The Ohio State University College
More informationA Hierarchical Visual Saliency Model for Character Detection in Natural Scenes
A Hierarchical Visual Saliency Model for Character Detection in Natural Scenes Renwu Gao 1(B), Faisal Shafait 2, Seiichi Uchida 3, and Yaokai Feng 3 1 Information Sciene and Electrical Engineering, Kyushu
More informationPre-Attentive Visual Selection
Pre-Attentive Visual Selection Li Zhaoping a, Peter Dayan b a University College London, Dept. of Psychology, UK b University College London, Gatsby Computational Neuroscience Unit, UK Correspondence to
More informationThere s Waldo! A Normalization Model of Visual Search Predicts Single-Trial Human Fixations in an Object Search Task
Cerebral Cortex, July 26;26: 364 382 doi:.93/cercor/bhv29 Advance Access Publication Date: 9 June 25 Original Article ORIGINAL ARTICLE There s Waldo! A Normalization Model of Visual Search Predicts Single-Trial
More informationElias B Issa, PhD. Curriculum Vitae 08/2016. RESEARCH INTEREST The neural mechanisms and computational principles underlying high-level vision
Elias B Issa, PhD Curriculum Vitae 08/2016 Massachusetts Institute of Technology 77 Massachusetts Ave, 46-6157 Cambridge, MA 02139 Office / 617.324.0306 Cell / 443.310.5896 Email / issa@mit.edu RESEARCH
More informationA quantitative theory of immediate visual recognition
A quantitative theory of immediate visual recognition Thomas Serre, Gabriel Kreiman, Minjoon Kouh, Charles Cadieu, Ulf Knoblich and Tomaso Poggio Center for Biological and Computational Learning McGovern
More informationOptimal Coding Predicts Attentional Modulation of Activity in Neural Systems
LETTER Communicated by Emilio Salinas Optimal Coding Predicts Attentional Modulation of Activity in Neural Systems Santiago Jaramillo sjara@ieee.org Barak A. Pearlmutter barak@cs.nuim.ie Hamilton Institute,
More informationSearch for a category target in clutter
to appear: Perception, 2004. Search for a category target in clutter MARY J. BRAVO, 1 HANY FARID 2 An airport security worker searching a suitcase for a weapon is engaging in an especially difficult search
More informationSingle cell tuning curves vs population response. Encoding: Summary. Overview of the visual cortex. Overview of the visual cortex
Encoding: Summary Spikes are the important signals in the brain. What is still debated is the code: number of spikes, exact spike timing, temporal relationship between neurons activities? Single cell tuning
More informationElias B Issa, PhD. Curriculum Vitae 10/2015
Elias B Issa, PhD Curriculum Vitae 10/2015 Massachusetts Institute of Technology 77 Massachusetts Ave, 46-6157 Cambridge, MA 02139 Office / 617.324.0306 Cell / 443.310.5896 Email / issa@mit.edu RESEARCH
More informationTHE ENCODING OF PARTS AND WHOLES
THE ENCODING OF PARTS AND WHOLES IN THE VISUAL CORTICAL HIERARCHY JOHAN WAGEMANS LABORATORY OF EXPERIMENTAL PSYCHOLOGY UNIVERSITY OF LEUVEN, BELGIUM DIPARTIMENTO DI PSICOLOGIA, UNIVERSITÀ DI MILANO-BICOCCA,
More information8:30 BREAKFAST 10:20 COFFEE BREAK - POSTERS
MIT Camp mpus us, Buil ildi ding 46-6-30 02, 43 Vas assar St reet et, Ca mbri ridg dge, MA 0213 9 http tp:/ ://s /sun uns. s.mit.ed edu 8:30 BREAKFAST BUILDING SCENE REPRESENTATIONS: COMPUTER VISION 9:00
More informationControl of Selective Visual Attention: Modeling the "Where" Pathway
Control of Selective Visual Attention: Modeling the "Where" Pathway Ernst Niebur Computation and Neural Systems 139-74 California Institute of Technology Christof Koch Computation and Neural Systems 139-74
More informationOverview of the visual cortex. Ventral pathway. Overview of the visual cortex
Overview of the visual cortex Two streams: Ventral What : V1,V2, V4, IT, form recognition and object representation Dorsal Where : V1,V2, MT, MST, LIP, VIP, 7a: motion, location, control of eyes and arms
More informationToward a More Biologically Plausible Model of Object Recognition. Minjoon Kouh
Toward a More Biologically Plausible Model of Object Recognition by Minjoon Kouh Submitted to the Department of Physics in partial fulfillment of the requirements for the degree of Doctor of Philosophy
More informationMethods for comparing scanpaths and saliency maps: strengths and weaknesses
Methods for comparing scanpaths and saliency maps: strengths and weaknesses O. Le Meur olemeur@irisa.fr T. Baccino thierry.baccino@univ-paris8.fr Univ. of Rennes 1 http://www.irisa.fr/temics/staff/lemeur/
More informationA Neurally-Inspired Model for Detecting and Localizing Simple Motion Patterns in Image Sequences
A Neurally-Inspired Model for Detecting and Localizing Simple Motion Patterns in Image Sequences Marc Pomplun 1, Yueju Liu 2, Julio Martinez-Trujillo 2, Evgueni Simine 2, and John K. Tsotsos 2 1 Department
More informationAttention and Scene Understanding
Attention and Scene Understanding Vidhya Navalpakkam, Michael Arbib and Laurent Itti University of Southern California, Los Angeles Abstract Abstract: This paper presents a simplified, introductory view
More informationComputational Saliency Models Cheston Tan, Sharat Chikkerur
Computational Salieny Models Cheston Tan, Sharat Chikkerur {heston,sharat}@mit.edu Outline Salieny 101 Bottom up Salieny Model Itti, Koh and Neibur, A model of salieny-based visual attention for rapid
More informationPosition invariant recognition in the visual system with cluttered environments
PERGAMON Neural Networks 13 (2000) 305 315 Contributed article Position invariant recognition in the visual system with cluttered environments S.M. Stringer, E.T. Rolls* Oxford University, Department of
More informationThe role of perceptual load in orientation perception, visual masking and contextual integration Moritz J.K. Stolte
The role of perceptual load in orientation perception, visual masking and contextual integration Moritz J.K. Stolte Institute of Cognitive Neuroscience University College London Submitted for the degree
More informationEDGE DETECTION. Edge Detectors. ICS 280: Visual Perception
EDGE DETECTION Edge Detectors Slide 2 Convolution & Feature Detection Slide 3 Finds the slope First derivative Direction dependent Need many edge detectors for all orientation Second order derivatives
More informationThe role of early visual cortex in visual integration: a neural model of recurrent interaction
European Journal of Neuroscience, Vol. 2, pp. 189 11, 24 ª Federation of European Neuroscience Societies The role of early visual cortex in visual integration: a neural model of recurrent interaction Gustavo
More informationThe Integration of Features in Visual Awareness : The Binding Problem. By Andrew Laguna, S.J.
The Integration of Features in Visual Awareness : The Binding Problem By Andrew Laguna, S.J. Outline I. Introduction II. The Visual System III. What is the Binding Problem? IV. Possible Theoretical Solutions
More informationA coincidence detector neural network model of selective attention
A coincidence detector neural network model of selective attention Kleanthis Neokleous1 (kleneokl@cs.ucy.ac.cy) Maria Koushiou2 (maria_koushiou@yahoo.com) Marios N. Avraamides2 (mariosav@ucy.ac.cy) Christos
More informationTitle: Cortical architecture for the organization of spatial and object attention
Editorial Manager(tm) JCNS Manuscript Draft Manuscript Number: Title: Cortical architecture for the organization of spatial and object attention Article Type: Manuscript Keywords: neural model, biased
More informationNeurobiological Models of Visual Attention
Neurobiological Models of Visual Attention John K. Tsotsos Dept. of Computer Science and Centre for Vision Research York University Theories/Models J.K.Tsotsos 2 Müller (1873) Exner (1894) Wundt (1902)
More informationOn the role of context in probabilistic models of visual saliency
1 On the role of context in probabilistic models of visual saliency Date Neil Bruce, Pierre Kornprobst NeuroMathComp Project Team, INRIA Sophia Antipolis, ENS Paris, UNSA, LJAD 2 Overview What is saliency?
More informationHuman Learning of Contextual Priors for Object Search: Where does the time go?
Human Learning of Contextual Priors for Object Search: Where does the time go? Barbara Hidalgo-Sotelo, Aude Oliva, Antonio Torralba Department of Brain and Cognitive Sciences and CSAIL, MIT MIT, Cambridge,
More informationAttentive Stereoscopic Object Recognition
Attentive Stereoscopic Object Recognition Frederik Beuth, Jan Wiltschut, and Fred H. Hamker Chemnitz University of Technology, Strasse der Nationen 62, 09107 Chemnitz, Germany frederik.beuth@cs.tu-chemnitz.de,wiltschj@uni-muenster.de,
More informationNeurally Inspired Mechanisms for the Dynamic Visual Attention Map Generation Task
Neurally Inspired Mechanisms for the Dynamic Visual Attention Map Generation Task Maria T. Lopez 1, Miguel A. Fernandez 1, Antonio Fernandez-Caballero 1, and Ana E. Delgado 2 Departamento de Informatica
More informationThe `Bayesian approach to perception, cognition and disease (3)
A Bayesian theory of the Brain - 1990s- Purpose of the brain: infer state of the world from noisy and incomplete data [G. Hinton, P. Dayan, A. Pouget, R. Zemel, R. Rao, etc..] The `Bayesian approach to
More informationGoals. Visual behavior jobs of vision. The visual system: overview of a large-scale neural architecture. kersten.org
The visual system: overview of a large-scale neural architecture Daniel Kersten Computational Vision Lab Psychology Department, U. Minnesota Goals Provide an overview of a major brain subsystem to help
More informationLateral Geniculate Nucleus (LGN)
Lateral Geniculate Nucleus (LGN) What happens beyond the retina? What happens in Lateral Geniculate Nucleus (LGN)- 90% flow Visual cortex Information Flow Superior colliculus 10% flow Slide 2 Information
More informationTime Course of Attention Reveals Different Mechanisms for Spatial and Feature-Based Attention in Area V4
Neuron, Vol. 47, 637 643, September 1, 2005, Copyright 2005 by Elsevier Inc. DOI 10.1016/j.neuron.2005.07.020 Time Course of Attention Reveals Different Mechanisms for Spatial and Feature-Based Attention
More informationIn: Connectionist Models in Cognitive Neuroscience, Proc. of the 5th Neural Computation and Psychology Workshop (NCPW'98). Hrsg. von D. Heinke, G. W.
In: Connectionist Models in Cognitive Neuroscience, Proc. of the 5th Neural Computation and Psychology Workshop (NCPW'98). Hrsg. von D. Heinke, G. W. Humphreys, A. Olson. University of Birmingham, England,
More informationKeywords- Saliency Visual Computational Model, Saliency Detection, Computer Vision, Saliency Map. Attention Bottle Neck
Volume 3, Issue 9, September 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Visual Attention
More informationPreparatory attention in visual cortex
Ann. N.Y. Acad. Sci. ISSN 0077-8923 ANNALS OF THE NEW YORK ACADEMY OF SCIENCES Issue: The Year in Cognitive Neuroscience REVIEW ARTICLE Preparatory attention in visual cortex Elisa Battistoni, 1 Timo Stein,
More informationSerial visual search from a parallel model q
Vision Research 45 (2005) 2987 2992 www.elsevier.com/locate/visres Serial visual search from a parallel model q Seth A. Herd *, Randall C. OÕReilly Department of Psychology, University of Colorado Boulder,
More informationAdventures into terra incognita
BEWARE: These are preliminary notes. In the future, they will become part of a textbook on Visual Object Recognition. Chapter VI. Adventures into terra incognita In primary visual cortex there are neurons
More informationVisual Attention and the Role of Normalization
Visual Attention and the Role of Normalization The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters. Citation Accessed Citable Link
More informationTop-down Attention Signals in Saliency 邓凝旖
Top-down Attention Signals in Saliency WORKS BY VIDHYA NAVALPAKKAM 邓凝旖 2014.11.10 Introduction of Vidhya Navalpakkam EDUCATION * Ph.D, Computer Science, Fall 2006, University of Southern California (USC),
More informationOscillatory Neural Network for Image Segmentation with Biased Competition for Attention
Oscillatory Neural Network for Image Segmentation with Biased Competition for Attention Tapani Raiko and Harri Valpola School of Science and Technology Aalto University (formerly Helsinki University of
More informationChanging priority maps in 12- to 18-month-olds: An emerging role for object properties
Psychonomic Bulletin & Review 1, 17 (1), - doi:1.375/pbr.17.1. Changing priority maps in 1- to 1-month-olds: An emerging role for object properties Ada m Sh e ya and Li n da B. Sm i t h Indiana University,
More informationVisual attention: the where, what, how and why of saliency Stefan Treue
428 Visual attention: the where, what, how and why of saliency Stefan Treue Attention influences the processing of visual information even in the earliest areas of primate visual cortex. There is converging
More informationObject and Gist Perception in a Dual Task Paradigm: Is Attention Important?
Object and Gist Perception in a Dual Task Paradigm: Is Attention Important? Maria Koushiou1 (maria_koushiou@yahoo.com) Elena Constantinou1 (elena_constantinou@yahoo.gr) 1 Department of Psychology, University
More informationCortical Algorithms for Perceptual Grouping
I ANRV278-NE29-07 ARI 7 March 2006 10:13 R E V I E W S E C N A D V A N Cortical Algorithms for Perceptual Grouping Pieter R. Roelfsema The Netherlands Ophthalmic Research Institute, Meibergdreef 47, 1105
More informationVisual Context Dan O Shea Prof. Fei Fei Li, COS 598B
Visual Context Dan O Shea Prof. Fei Fei Li, COS 598B Cortical Analysis of Visual Context Moshe Bar, Elissa Aminoff. 2003. Neuron, Volume 38, Issue 2, Pages 347 358. Visual objects in context Moshe Bar.
More informationNIH Public Access Author Manuscript Vis cogn. Author manuscript; available in PMC 2009 December 8.
NIH Public Access Author Manuscript Published in final edited form as: Vis cogn. 2009 August 1; 17(6-7): 945 978. doi:10.1080/13506280902834720. Modeling Search for People in 900 Scenes: A combined source
More informationFace Neurons. Chapter 4. Edmund T. Rolls. The discovery of face neurons
Chapter 4 Face Neurons Edmund T. Rolls The discovery of face neurons In 1976 we recorded from single neurons in the awake behaving non-human primate to investigate reward systems in the brain. We had shown
More informationVision Research 49 (2009) Contents lists available at ScienceDirect. Vision Research. journal homepage:
Vision Research 49 (29) 29 43 Contents lists available at ScienceDirect Vision Research journal homepage: www.elsevier.com/locate/visres A framework for describing the effects of attention on visual responses
More informationReport. Attention Narrows Position Tuning of Population Responses in V1
Current Biology 19, 1356 1361, August 25, 2009 ª2009 Elsevier Ltd All rights reserved DOI 10.1016/j.cub.2009.06.059 Attention Narrows Position Tuning of Population Responses in V1 Report Jason Fischer
More informationIN this paper we examine the role of shape prototypes in
On the Role of Shape Prototypes in Hierarchical Models of Vision Michael D. Thomure, Melanie Mitchell, and Garrett T. Kenyon To appear in Proceedings of the International Joint Conference on Neural Networks
More informationAttention, uncertainty, and free-energy
HUMAN NEUROSCIENCE Original Research Article published: 0 December 010 doi: 10.3389/fnhum.010.0015 Attention, uncertainty, and free-energy Harriet Feldman and Karl J. Friston* The Wellcome Trust Centre
More informationChapter 3: 2 visual systems
Chapter 3: 2 visual systems Overview Explain the significance of the turn to the brain in cognitive science Explain Mishkin and Ungerleider s hypothesis that there are two distinct visual systems Outline
More informationObject detection in natural scenes: Independent effects of spatial
DOI 10.3758/s13414-017-1279-8 Object detection in natural scenes: Independent effects of spatial and category-based attention Timo Stein 1,2 & Marius V. Peelen 1 # The Author(s) 2017. This article is published
More informationA Source for Feature-Based Attention in the Prefrontal Cortex
A Source for Feature-Based Attention in the Prefrontal Cortex The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published
More informationThe probabilistic approach
Introduction Each waking moment, our body s sensory receptors convey a vast amount of information about the surrounding environment to the brain. Visual information, for example, is measured by about 10
More informationHuman Learning of Contextual Priors for Object Search: Where does the time go?
Human Learning of Contextual Priors for Object Search: Where does the time go? Barbara Hidalgo-Sotelo 1 Aude Oliva 1 Antonio Torralba 2 1 Department of Brain and Cognitive Sciences, 2 Computer Science
More informationTrade-Off between Object Selectivity and Tolerance in Monkey Inferotemporal Cortex
12292 The Journal of Neuroscience, November 7, 2007 27(45):12292 12307 Behavioral/Systems/Cognitive Trade-Off between Object Selectivity and Tolerance in Monkey Inferotemporal Cortex Davide Zoccolan, Minjoon
More informationNeural Mechanisms Underlying Visual Object Recognition
Neural Mechanisms Underlying Visual Object Recognition ARASH AFRAZ, DANIEL L.K. YAMINS, AND JAMES J. DICARLO Department of Brain and Cognitive Sciences and McGovern Institute for Brain Research, Massachusetts
More information