1 Perception& Attention. 5 The Retina. 3 Overview of the Visual System. 6 LGN of the Thalamus. 4 Two Streams: Ventral what vs.

Transcription

1 1 Perception& Attention Perception is effortless but its underlying mechanisms are incredibly sophisticated. Biology of the visual system Representations in primary visual cortex and Hebbian learning Object recognition Attention: Interactions between systems involved in object recognition and spatial processing 2 Perception& Attention Some motivating questions: 1. Why does primary visual cortex encode oriented bars of light? 2. Why is visual system split into what/where pathways? 3. Why does parietal damage cause attention problems (neglect)? 4. How do we recognize objects (across locations, sizes, rotations with wildly different retinal images)? 3 Overview of the Visual System 4 Two Streams: Ventral what vs. Dorsal where Hierarchies of specialized visual pathways, starting in retina, to LGN (thalamus),to &up: PO V3A m m VIP MST PG right field temporal optic chiasm V2,V4... p MT FST left field nasal temporal LGN V2,V4... d V2 V3 V4 TEO PG TE TF TE 5 The Retina Retina is not a passive camera Key principle: contrast enhancement that emphasizes changes over space & time. a) Oncenter b) Offcenter retinal output ganglion cells 6 LGN of the Thalamus A relaystation, butsomuch more. Organizes different types of information into different layers. Performs dynamic processing: magnocellular motion processing cells, attentional processing. On- and off-center information from retina is preserved in LGN

2 7 Primary Visual Cortex (): Edge Detectors combines LGN (thalamus) inputs into oriented edge detectors: a) Oncenter b) Offcenter Edges differ in orientation, size (spatial frequency), and position. 8 Primary Visual Cortex (): Edge Detectors combines LGN (thalamus) minputs into oriented edge detectors: a) Oncenter b) Offcenter Edges differ in orientation, size (spatial frequency), and position. edge detector For coherent vision, need to detect varying degrees of all these. For coherent vision, need to detect varying degrees of all these. 9 Primary Visual Cortex (): Topography hypercolumns 10 Rerouting of Visual Info to Auditory Cortex Sharma, Angelucci & Sur(2000), Nature Rerouted fibers from Retina auditory thalamus(mgn) A1 blobs L R L occularity R orientations Pinwheel Hypercolumn: Full set of coding for each position Pinwheel can arise from learning and lateral connectivity: not hard-wired! If visual properties are learned, they should develop in A1. 11 Rerouting of Visual Orientation Modules in A1 12Visual Behavior After Rerouting Right Visual Field von Melchner, Pallas & Sur (2000) Ba-d: Orientation maps, dark- high act for given orientation (bottom right). C: composite map of orientation preferences D: red dots=pinwheel centers

3 13 Visual Acuity After Rerouting So learning is powerful, but so is evolution! 14 A Question What makes visual cortex visual cortex? Why does it represent oriented bars of light? 15 Primary Visual Representations 16 The Model: Simulating one Hypercolumn Key idea: Oriented edge detectors can develop from Hebbian correlational learning based on natural visual scenes. Natural visual scenes are preprocessed by passing them(separately) through layers of on-center and off-center inputs Hiddenlayer: edgedetectors seen in layers2/3 of; Layer4(input)just represents unoriented on/off inputs like LGN (but can be modulated by attention) 17 The Model: Simulating one Hypercolumn 18 [v1rf.proj] Hebbian learning only KWTA inhib competition for specialization (see Ch 4)

4 19 The Receptive Fields Red = on-center > off-center, Blue = off-center > on-center Perception and Attention 1. Why does primary visual cortex encode oriented bars of light? Correlational learning based on natural visual scenes. Reflects reliable presence of edges in natural images, which vary in size, position, orientation and polarity. model shows how documented properties can result from interactions between learning, architecture (connectivity), and structure of environment. Some differences, but pinwheels still emerge

5 25 Perception and Attention 1. Why does primary visual cortex encode oriented bars of light? Correlational learning based on natural visual scenes. 3. Why is visual system split into what/where pathways? 4. Why does parietal damage cause attention problems (neglect)? 26 The Object Recognition Problem Problem: Recognize object regardless of: location, size, rotation. Same Diff This is hard because different patterns in same location can overlap a lot, while the same patterns in different locations/sizes/rotations can notoverlap atall! 2. How do we recognize objects (across locations, sizes, rotations with wildly different retinal images)? 27 28Gradual Invariance Transformations (Fukushima, 80) Increasing receptive field size enables: Conjunction of features(to form more complex objects); and Collapsing over location information ( spatial invariance ) 29Gradual Invariance Transformations (Fukushima, 80) 30Gradual Invariance Transformations (Fukushima, 80) if did spatialinvariance in onefell swoop: binding problem-can ttell T froml Goal: Units at the top of the hierarchy should represent complex object features in a location and size invariant fashion (also want benefits of top-down amplification, pattern completion, distributed reps etc)

6 31 The Model V2 LGN_On V4/IT Output LGN_Off = oriented line (edge) detectors, hard-coded V2 unitsencode conjunctions of edgesacross a subset ofspace Each V4 unitpaysattention toall ofv2 32 The Objects Each object is presented at multiple locations, sizes Network s job is to activate the appropriate Output unit(0-19) for each object, regardless of location and size 33 [objrec.proj]

7

8 43 44 Generalization Can the network generalize to unseen views of studied objects? Inother words: Does trainingthe netto recognize a set of objects in a size/location invariant fashion help it recognize new objects in a size/location invariant fashion? Procedure: Takeanettrained on 18objects Train with 2 new objects in only some locations/sizes Test the net with nonstudied views (sizes/locations) of new objects Generalization Can the network generalize to unseen views of studied objects? yes Approx. 75% correct on novel views following training on 10% of possible sizes/locations Explanation: Distributed representations and Hebb learning! V4 represents object features in a location/size invariant way Each object activates a distributed pattern of these invariant feature detectors 47 48

9 Yeah, but these objects are regularly shaped, straight lines... what about real objects?

10 55 56 Emer the robot recognizing objects.. 57 Video Demo O Reilly, CogSci A Challenge why? bidir cons support attractors across multiple levels of net to amplify consistent info

11 60 62 State of the Art Perception and Attention 1. Why does primary visual cortex encode oriented bars of light? Correlational learning based on natural visual scenes. 2. How do we recognize objects (across locations, sizes, rotations with wildly different retinal images)? Transformations: increasingly complex featural encodings, increasing levels of spatial invariance; Distributed representations. 3. Why is visual system split into what/where pathways? 4. Why does parietal damage cause attention problems (neglect)? Motion Still missing... Neurons in area MT very sensitive to motion Lots of work on how downstream areas integrate motion signals across time to detect coherence (e.g. Shadlen, Newsome, etc) Thomas Serre has shown that motion signals very reliable for discriminating between particular actions (eg throwing a baseball) Patient copying a Spatial Attention: Unilateral Neglect Should be able to solve problem via bidirectional influence of motion integration signals, object recognition, and spatial attention (next) scene Self portrait, copying, line bisection tasks: In all cases, patients with parietal/temporal lesions seem to forget about 1/2 of space! but they still see it! 65

12

13 Effects of Parietal Lesions on Posner Task 75 Possible Models Lesioned Alert 80 Interrupt 60 Intact 40 Localize Spatial Object Neutral Valid Invalid Patients perform normally in the neutral (no cue) condition, regardless of where the target is presented Patients benefit just as much as controls from valid cues Disengage Move Engage Inhibit (features x location) Patients are hurt more than controls by invalid cues Attention emerges from bidirectional constraint satisfaction& inhibitory competition. 76 Simple Model 77 [attn simple.proj] targ Spat2 cue Output Obj2 Spat1 Obj1 Object 2 (Target) Input Object 1 (Cue)

14 78 Posner Task Data Valid Invalid Diff Adult Normal Elderly Normal Patients Elderly normalized (*.65) Patients normalized (*.55) Posner Task Sims The model explains the basic finding that valid cues speed target processing, while invalid cues hurt Also explains finding that patients with small unilateral parietal lesions benefit normally from valid cues in ipsilateral field but are disproportionately hurt by invalid cues. No need to posit disengage module! Also explains finding of neglect of contralateral visual field after large, unilateral parietal lesions when some stimulus is present in ipsilateral field( extinction ) 80 More Posner Lesion Fun 81 More Posner Lesion Fun Returning to patient with left parietal lesion... What happens if cues are presented in contralateral(affected) hemifield? ( Reverse Posner ) Returning to patient with left parietal lesion... What happens if cues are presented in contralateral(affected) hemifield? Predictions: Smaller benefit for valid cues Patients should be hurt less than controls by invalid cues. 82 [attn simple.proj] 83 Inhibition of Return Typically, target detection is faster on trials with valid vs invalid cues However,if thecue ispresented for a longer time(eg. 500ms), performance is faster on invalid vs valid trials Can explain in terms of accommodation(neural fatigue)

15 84 [attn simple.proj] 85 Simple model: too simple? Has unique one-to-one mappings between low-level visual features and object representations (not realistic) Does not address issue of spatial attention when trying to perceive multiple objects simultaneously Complex model combines more realistic model of object recognition (starting from LGN) with simple attention model Can use spatial attention to restrict object processing pathwaytoone object atatime,enabling it tosequentially process multiple objects. Lesions of entire spatial pathway cause simultanagnosia: inability to concurrently recognize two objects 86 Complex Model 87 Perception and Attention Spat1 Spat2 V4/IT Output Target 1. Why does primary visual cortex encode oriented bars of light? Correlational learning based on natural visual scenes. V2 2. How do we recognize objects (across locations, sizes, rotations with wildly different retinal images)? Transformations: increasingly complex featural encodings, increasing levels of spatial invariance; Distributed representations. LGN_On LGN_Off Spat1 has recurrent projns to encourage focus on one region of space But only mechanism for switching is accommodation Why is visual system split into what/where pathways? Transformations: emphasizing and collapsing across different types of relevant distinctions 4. Why does parietal damage cause attention problems (neglect)? Attention as an emergent property of competition 88 General Issues in Attention Attention: Prioritizes processing. Coordinates processing across different areas. Solves binding problems via coordination. But attention should be much more flexible than just spatial bias! Later: how to incorporate goals, reinforcement probability, into attentional allocation