Models of Models of predictive: can we predict eye movements (bottom up attention)? [L. Itti and coll] pop out and saliency? [Z. Li] Readings: Maunsell & Cook, the role of attention in visual processing, 22. Itti & Koch, Computational modeling of visual attention, Nat Rev Neurosci, 21. phenomenological : a simple unifying description? [Reynold and Heeger, 29] mechanistic : which circuits? [Z. Li, A. Compte] functional : deriving neural effects based on a computational principle? [Jaramillo and Pearlmutter] functional & Bayesian : can we think of attention as a prior? what is the role of attention in representing uncertainty? [Rao, Yu and Dayan] M. C. Escher, Balcony, 1945 Bottom-up : Saliency Maps and eye movements Itti s model and Prediction of eye movements What determines which stimuli are selected by the attentional process and which will be discarded? idea: There is a representation of a saliency map Saliency at a given location = how different this location is from its surround in color, orientation, motion, depth etc (Koch and Ullman, 1985, Itti et al. 1998) Selection in the map by Winner-Take-All mechanism. Followed by suppression of chosen location (`inhibition of return!) leading to selection of 2d highest value as next location. Alternative approaches to saliency : Bayesian surprise measure (e.g. Itti and Baldi 25 CVPR), and information-theoretic measures (e.g. Bruce and Tsotsos 26 NIPS; Najemnik and Geisler 25 Nature). http://ilab.usc.edu/
Itti s model and Prediction of eye movements Circuits of Bottom-up : V1 as a Saliency Map Anatomical localization of the saliency map(s)? Z. Li (22): V1 as a saliency map -- firing rates proportional to saliency. Effort to link anatomy, physiology (center-surround modulations) and psychophysics other applications: surveillance, automatic target detection, navigational aids and robotic control.&/)#12( "+$*+$(3%"4(,-(4"5/,-(4"5/,-("+$*+$' Highlighting important image locations, where translation invariance in inputs breaks down. 6(%1+%%#$(#$7"%8(7)$9(:#$%&; 1"%$)1&/(:#$&1$)"#'($9&$(<1+$'( 1"#$<$+&/()#3/+#1' http://ilab.usc.edu/!"#$%&'$( )#*+$($"(,- Mechanisms of Bottom-Up P. Series et al. / Journal of Mechanisms of Bottom-Up Similarly the responses of V1 neurons are modulated by stimuli presented outside the receptive field. surround suppression, collinear facilitation, cross facilitation. Series et al, 23 The perception of an object depends on its surroundings (and its saliency) -- contextual modulations. Contour integration, segmentation, pop out. Series et al, 23
Mechanisms of Bottom-Up Mechanisms of Top-down : Ardid et al (27) Modeling the circuits involved in top down attention and making assumptions about top down signals Identify plausible circuit mechanisms that underlie gain modulation, scaling of tuning curve, biased competition. Ardid et al. Microcircuit Model of al Processing Series et al, 23 Long range horizontal connections in V1 are supposed to be involved. Numerous Modelling studies to relate the circuits, neural recordings, and psychophysics. Figure 2. Feature-based attention in model simulations for single motion stimulus. A, Network activity for an unattended (left) and an attended trial (right). x-axis, Time; y-axis, neurons labeled by preferred direction pref. Activity is color-coded. C, Cue period; D, delay period; T, test period. Calibration, 1 s. B, Activity of a neuron with pref S. Top, Sample membrane potential; middle,spiketrainsinseveraltrials;bottom,trial-averagedactivity(red,attended;black,unattendedtrials)(calibration:time,1s; voltage, 5 mv; rate, 4 Hz). C, Selective enhancement of MT population activity. The scheme (top) depicts how the curves (attended in red; unattended in black) were generated: for fixed test stimulus S and attended feature A, the activity of all neurons (blue arrows) were measured. D, Smoothed modulation ratio (firing rate with attention divided by that without atten- The Normalization Model of Reynolds & Heeger, 29: a single, unifying computational principle / phenomenological model of (top down) attention Aimed at reconciling seemingly contradictory claims: whether attention is a contrast gain, or response gain and whether it scales tuning curves, or changes width. Normalization model Linear weighting function Division Retinal image R(x, θ) = Rectification E(x, θ) s(x, θ) E(x, θ) + σ Firing rate Extension on previous model Reynolds, Pasternak, & Desimone, Neuron, 2 Williford & Maunsell, J Neurophysiol, 26 Other cortical cells A Firing Rate (Hz) B 4 35 3 25 2 15 1 5 al modulation (%) Firing Rate (Hz) 66 C 8 33 % al Modulation 1 5 2 al modulation (%) normalized response Heeger, Vis Neurosci, 1992 = unnormalized response unnormalized + "! responses Normalization model: responses of neurons are normalized by a signal that represents the mean activity of the local pool of neurons Explains contrast saturation, and many nonlinear suppressive effects observed in physiology.
Surround suppression & normalization Normalization model of attention Optimal diameter Surround diameter Response (imp/sec) 8 6 4 2 452l21.p5 1 2 3 4 5 6 Grating patch diameter (deg) Suppression CRF Surround kc Stimulus Excitatory field Excitatory drive al field X Response k s CRF Suppressive field Surround Cavanaugh, Bair & Movshon, J Neurophysiol, 22 Normalization model was extended to account for surround suppression the normalization pool creates a suppressive drive around receptive field (stimulus drive) Suppressive drive Idea: attentional field (whose size depends on subjects! strategy) modulates the excitatory field, before it is normalized. R(x, θ) = A(x, θ)e(x, θ) s(x, θ) A(x, θ)e(x, θ) + σ Small stimulus, large attention field Predominantly Contrast Gain 1 1 A Normalized Model Response al Modulation (%) c c + " " c " c + # =! c c + # / "! > 1 attentional gain affects excitatory drive and suppressive drive equally Log Contrast 15 Modulation 16
Large stimulus, small attention field Predominantly Response Gain 1 1 B Normalized Model Response Log Contrast al Modulation (%) c c + "c + # For c >>! " c " c + #c + $! > 1 attentional gain 1 1 + " For c >>! " " + # <! < 1 surround suppression Modulation Spatial attention scales tuning curve McAdams & Maunsell, J Neurosci, 1999 A B C 1. Normalized Response One stimulus within receptive field and the other contralateral..5. R(c,!) = "R(c)R(!) Normalized Response Model Separable function of contrast and orientation; change in response gain or contrast gain affects only R(c). Featural attention can sharpen tuning curves Response 4 Treue & Martinez-Trujillo, Nature, 1999 Attended Ignored Normalized Response Model Functional role of Gain Modulations: Fisher Information If adaptation is about increasing Fisher Information at the attended location, what should it do to tuning curves? Nakahara and Amari (21) Mathematical analysis -- optimization of FI with constraints : baseline activity and height of tuning curves increase together. predicts both increases and decreases of tuning curves amplitude Two stimuli moved same direction on each trial, one within receptive field, and other in opposite hemifield. Attend fixation or attend same motion in opposite hemifield. R(!) = "A(!)E(!) field Excitatory drive
!! Functional role of Gain Modulations: Task dependency If attention is about optimizing the task, the shape of gain modulations should vary with the task Reconstruction task (Jaramillo & Pearlmutter) Estimation Broad vs Fine Discrimination Estimation theory + constraints to derive optimal modulations. Functional role of Gain Modulations : Dependency on the Read-out s Adaptation! attention State! Population Response!! Encoder r Fixed! Decoder Decoder ŝ ( 1) P new (r s) Series et al 29 Optimization strategies and predictions for perception depend on the nature of the read-out If the read-out is unaware of the attentional modulation, the modulation is bound to produce changes in appearance (estimation) and in discrimination.