Image Processing in the Human Visual System, a Quick Overview

Image Processing in the Human Visual System, a Quick Overview By Orazio Gallo, April 24th, 2008

The Visual System Our most advanced perception system: The optic nerve has 106 fibers, more than all the fibers inserting in the spine. The acoustic nerve has 30 x 103 fibers. In macaques almost 50% of the cortex is involved in vision, slightly less than that for humans.

The Retina is Part of the Central Nervous System The retina has the same neurotransmitters as the brain and stems directly from it.

The Retina is Part of the Central Nervous System

Why Three Types of Cones? Question: if we only had the R cones, would we be able to see an object and say with confidence if it is or it is not red?

Why Three Types of Cones? Question: if we only had the R cones, would we be able to see an object and say with confidence if it is or it is not red? NO!! The curves overlap so an intense blue light can stimulate an R cone as much as a faint red light! Color vision is NOT used to distinguish small details (multiple cones are required!)

Why Three Types of Cones?

The Retina

The Retina Signals are discrete (action potentials) only from the interneurons on! Neurons are roughly organized in 3 layers, receptors, interneurons, ganglion cells.

The Retina is an Active Sensor Contrast carries more information (e.g. contours of objects.) These cells help the higher systems to detect fast changes in pictures, particularly in lowcontrast conditions.

The Retina Different types of ganglion cells: M (magni): large receptive fields and sensitive to transient illumination (large objects and moving stimuli?) P (parvi): small receptive fields, tuned for particular wavelengths, more numerous (small objects and color vision?) Other (mostly unknown) types.

The Retina Bipolar, horizontal, and amacrine: Contribute to the creation of the ganglion's receptive fields. Bipolar (vertical pathway) are central, horizontal and amacrine (lateral pathway) are peripheral.

The Hermann Grid Illusion Explained

Central Vision Pathways

Central Vision Pathways The cells in the pretectal area receive information from the ganglion cells and control the pupillary reflex.

Central Vision Pathways The superior colliculus coordinates the head and the saccadic movements to direct the foveas towards the stimuli (it has access to information about sound localization and body position.) It also projects to the visual cortex (indirect pathway.)

Central Vision Pathways The Lateral Geniculate Nucleus (LGN) receives 90% of the fibers from the retina.

The LGN Six physically separated layers with circular, both on-center and off-center receptive fields. One magnocellular layer (movement and rought image characterization) and two parvocellular layers (details and shape + colors).

The Visual Cortex 17: V1 18: V2 19: V3 Brodmann (1905)

The Visual Cortex: V1 The on-center and off-center cells in the LGN are combined to form: Simple cells (edge detectors) Complex cells (line detectors)

The Visual Cortex: V1 Simple and complex cells are organized in columns and hyper-columns

What About Color? Chromatic opponence: R vs G, Y vs B, W vs B. A mix of V and B gives violet.. A mix of R and G gives yellow and all the traces of R and G are lost.

What About Color? The P ganglion cells in the retina and in the LGN:

What About Color? Double chromatic opponence: The margins of a gray object on a red background look greenish The margins of a gray object on a green background look reddish

What About Color? Double opponence cells in V1:

What About Color? Color constancy Colors are relatively constant to changes of illumination and spectral composition of the light

What About Color? In V1 the cells sensitive to color are organized in blobs. The receptive fields become more complex in terms of the combination of colors and the position of the afferent receptors.

What and Where Pathways The visual system has pathways that do concurrent processing (form, color, depth, and motion.) The dorsal pathway is the Where pathway The ventral pathway is the What pathway

Techniques for Analysis Single-cell recordings Auto-radiography Brain imaging: Computer Tomography (CT) Positron Emission Tomography (PET) Magnetic Resonance Imagery (MRI) Functional MRI

Techniques for Analysis Single-cell recordings Auto-radiography Brain imaging: Computer Tomography (CT) Positron Emission Tomography (PET) Magnetic Resonance Imagery (MRI) Functional MRI Show video from the Carandini lab

Techniques for Analysis Single-cell recordings Auto-radiography Brain imaging: Computer Tomography (CT) Positron Emission Tomography (PET) Magnetic Resonance Imagery (MRI) Functional MRI

Voltage-Sensitive Dye Imaging Benucci A., Frazor, R. A., and Carandini M. Standing Waves and Traveling Waves Distinguish Two Circuits in Visual Cortex, Neuron 2007

Voltage-Sensitive Dye Imaging Show prediction video courtesy of A. Benucci and M. Carandini

Thanks! Unless differently stated, the pictures are adapted from: Eric R. Kandel, James H. Schwartz, Thomas M. Jessell, Principles of Neuroscience, McGraw Hill, 2000. Stephen E. Palmer, Vision Science, Photons to Phenomenology, MIT press, 2002. The 3D picture of the brain in the top right corner of some slides is adapted from: http://www.inf.ufrgs.br/~carla/

Backup Slide: A Neuron