Neural codes PSY 310 Greg Francis Lecture 12 Is 100 billion neurons enough? COC illusion The COC illusion looks like real squares because the neural responses are similar True squares COC squares Ganglion cell Purdue responses University 1
Ventral stream From visual cortex to the temporal lobe Involved in recognizing or identifying objects Critical features Cells in inferotemporal cortex respond to complex features in stimuli Not easy to measure neurophysiologically Why such bizarre patterns? 2
Real stimuli Following the ideas about the COC stimulus, our perceptual experience is determined by the neural responses to stimuli Suppose you see this picture of a cat Cell responses This IT cell might respond strongly And the other cells hardly at all 3
Cell responses One might be tempted to suggest that this cell codes the perceptual experience of the cat But this would not be correct This cell would respond similarly to many different types of stimuli But our perceptual experience is quite different! Neurophysiology and perception Does this mean that our perceptual experience is not determined by neural responses? After all similar neural responses should give rise to similar percepts No. Similarity between ganglion responses and perceptual experience works for the ganglion cells because of their anatomical location Everything else in the visual system is based on their responses 4
Neurophysiology and perception The rest of visual perception does not depend on the responses of this IT cell Neurophysiology and perception The rest of visual perception does not depend on the responses of this IT cell In fact, probably thousands of IT cells respond to this stimulus 5
Neurophysiology and perception The rest of visual perception does not depend on the responses of this IT cell In fact, probably thousands of IT cells respond to this stimulus Millions of ganglion cells Neurophysiology and perception The rest of visual perception does not depend on the responses of this IT cell In fact, probably thousands of IT cells respond to this stimulus Millions of ganglion cells Millions of orientation sensitive cells 6
Neurophysiology and perception The rest of visual perception does not depend on the responses of this IT cell In fact, probably thousands of IT cells respond to this stimulus Millions of ganglion cells Millions of orientation sensitive cells Dorsal stream Representation of objects Objects are unlikely to be represented by a single neuron Consider faces 7
Representation of objects Objects are unlikely to be represented by a single neuron Consider faces What do you do here? Representation of objects Objects are unlikely to be represented by a single neuron Consider faces What do you do with different expressions? What do you do here? 8
Representation of objects Objects are unlikely to be represented by a single neuron Consider faces Since we perceive and recognize all of these faces as being different. There must be a different neural representations. What do you do here? Distributed code Each cell codes some feature of the image 9
Distributed code Each cell codes some feature of the image The representation of a face is a pattern across the features. No two faces have the same pattern, so not the same percept. Distributed code Each cell codes some feature of the image Change of expression may change only some of the features. Similar faces have similar patterns; so similar percepts. 10
Features So what are the features? A difficult question to answer They need not be things that we would name Nose, eye, mouth, hair Could be Fourier components Or something completely different Fourier features Selecting certain Fourier components as features might not correspond to anything that we would name in the image 11
Distributed code How does it all get put together? We don t see features, we see faces. Related issue We earlier discussed how we can consider the visual system to consist of many retinotopic layers of activities from cells tuned to different features Orientation cells Ganglion cells Retina Image 12
Related issue There are actually many layers we never got a chance to talk about Color, size, motion, depth, texture, many more To an extent, they process things independently - features Motion Color Orientation cells Ganglion cells Retina Image Related issue If you see a red car go speeding by, you gets lots of responses from different parts of the brain Which part is the percept? How do they get coordinated? Motion Color Orientation cells Ganglion cells Retina Image 13
Related issue For that matter, if you are thinking of something else or listening intently to something You may not perceive the car Even though the nervous system responds! Motion Color Orientation cells Ganglion cells Retina Image Attention It s not entirely clear what attention is or does But it seems to be involved in pulling together neural responses from different parts of the brain Perhaps by synchronizing action potentials Perhaps it selects the features that help solve a particular task E.g., recognize a face Attention seems to be necessary to actually perceive something 14
R W S Prof. Greg Francis Attentional blink Suppose you have to identify rapidly presented (100 ms) letters e.g., detect and/or K in a stream of letters M P K Attentional blink Turns out that detection of first letter tends to make detection of the second letter very difficult if it immediately follows the first Attentional blink M P K R S 15
Attentional blink Measure frequency of detection Implies that detecting first letter causes you to miss second letter! Attentional blink Maybe when the appears, some cell in IT detects the curve This suggests it might be a But it could be an O, U or S S L N B R K H... Motion Ganglion cells Retina Image Color Orientation cells 16
Attentional blink This cell sends signals back down to other areas to augment some features S L N B R K H... Motion Ganglion cells Retina Image Color Orientation cells Attentional blink This leads to strong activation of other cells in IT Which overall producse the pattern that corresponds to a S L N B R K H... Motion Ganglion cells Retina Image Color Orientation cells 17
Attentional blink The process takes time and may not be finished before the disappears May continue even while other letters go by Including the K Attentional blink S L N B R K H... Motion Ganglion cells Retina Image Color K Orientation cells Conclusions Distributed coding Features Neural patterns Attention to bring it all together Attentional blink 18
Next time Review for the exam You ask questions I answer questions Exam 1 on Friday 19