Sensory Encoding. What do sensory systems do with the endless stream of stimuli? Eero Simoncelli. discard store respond. Encode. Stimulus.

Transcription

1 Sensory Encoding Eero Simoncelli 6 March 212 What do sensory systems do with the endless stream of stimuli? discard store respond Encode Stimulus Neural response Behavior ^ ^ P(v 1 > v 2 ) 1 % "v1 seen faster" % v1 psychometric function Encoding Transform and represent sensory information Decoding Etract encoded information for estimation/decision/ action

2 tuning curve response (sp/s) stimulus (orientation) + simple to measure - very incomplete: - limited stimuli - limited response model (no noise, spikes, dynamics) a black bo [Hubel 95] Problems with black boes... Curse of dimensionality Mimics, but doesn t eplain how or why Doesn t make predictions => Need a model

3 Some big numbers: seconds since big bang: atoms in the visible universe: 1-bit images, 11: ,, we can t enumerate them => we need models Model constraints Implementation (anatomy, biophysics, etc) Functional physiological properties Fundamental computational principles - Encoding efficiency An - algorithm Performance is likely in a task to be understood more readily by Occam s understanding Razor the nature of the problem being solved than by eamining the mechanism in which it is embodied. - Marr, 1982 Stimulus Neural response Behavior ^ ^ P(v 1 > v 2 ) 1 % "v1 seen faster" % v1 psychometric function Encoding Efficient coding hypothesis [Barlow 61] Decoding Optimal estimation/ decision Both theories rely on statistical models of environment

4 Common frameworks Science: optimality principles for neurobiology (evolution, development, learning, adaptation) Engineering: compression, denoising, restoration, enhancement/modification, synthesis, manipulation Neural characterization Efficient Coding [Attneave 54; Barlow 61; Laughlin 81; Atick 9; Bialek etal 91] Transform Maimize information about stimulus in response, subject to constraints (e.g. metabolic) edients: I(r, s) = H(r) H(r s) stimuli response model estimation method Efficiency in single neurons Utilize full response range Subject to constraints Subject to noise

5 Efficiency in single neurons response! cost! C(r) output! p(n r) input! p(n s) prior! p(s) Efficiency in single neurons response! cost! C(r) output! p(n r) input! p(n s) prior! p(s) Efficiency in single neurons response! cost! C(r) output! p(n r) input! p(n s) prior! p(s)

6 Efficiency in single neurons response! cost! C(r) output! p(n r) input! p(n s) prior! p(s) Efficiency in single neurons response! cost! C(r) output! p(n r) input! p(n s) prior! p(s) Measure contrast in natural scenes Construct cumulative PDF Compare to neural response (fly, large monopolar neuron) [Laughlin, 1981]

7 Current Biology, Vol. 13, , March 18, 23, 23 Elsevier Science Ltd. All rights reserved. The Cost of Cortical Computation DOI 1.116/S (3)135- Peter Lennie* rat neocorte. Neurons in human neocorte are larger Center for Neural Science than those in rat and receive and make more synapses, New York University but they are not otherwise known to differ in their basic 4 Washington Place structure or organization [5]. Thus, with appropriate New York, New York 13 scaling of parameters for the larger neurons, Attwell and Laughlin s analysis can be used to estimate the energy consumed by a pyramidal neuron in human neocorte. Summary In different mammals, the number of neurons under a unit area of cortical surface is relatively constant Electrophysiological recordings show that individual ( 1,/mm 2 ), ecept in primate striate corte, where neurons in corte are strongly activated when enan increase in cortical thickness and a proportionately it may be twice as high [6]. Increasing brain size brings gaged in appropriate tasks, but they tell us little about how many neurons might be engaged by a task, which lower density of neurons [5, 6] without an increase in is important to know if we are to understand how cell body size, which remains approimately constant at corte encodes information. For human corte, I estiincreases with cortical thickness. This reflects an in- 15 m diameter [7]. The volume of aons and dendrites mate the cost of individual spikes, then, from the known energy consumption of corte, I establish how crease in the lengths of dendrites and aons without an many neurons can be active concurrently. The cost of increase in diameter [5]. Table 1 summarizes relevant a single spike is high, and this severely limits, possibly statistics for human corte. to fewer than 1%, the number of neurons that can be substantially active concurrently. The high cost of Postsynaptic Potentials spikes requires the brain not only to use representa- Individual synapses are assumed to be the same in rat tional codes that rely on very few active neurons, but and human neurons, so the energy costs associated also to allocate its energy resources fleibly among with transmitter uptake and release will be the same, cortical regions according to task demand. The latter as will the current flow through receptor channels. Given constraint eplains the investment in local control of (from Table 1) synapses per mm 3 of corte, hemodynamics, eploited by functional magnetic res- and 4, neurons/mm 3, the average neuron will make onance imaging, and the need for mechanisms of se- 17,5 synaptic contacts. If we use this number, and lective attention. assume a 5% failure rate [8, 9], the cost of EPSPs arising from a single spike will be ATP molecules Introduction [4]. CSH-2 Efficiency in multiple neurons Utilize joint response range => independent Subject to constraints Subject to noise Simple approimation (low noise): responses should be statistically independent Efficient coding in the retina?

8 Retinal ganglion cell receptive fields ON OFF [Kuffler, 1953] Efficient coding in the retina? [Srinivasan et. al. 82; Atick & Redlich 9; van Hateren 92] noise noise s + filter + r whitening filter low-pass filter optimal filter from A [Atick & Redlich 92]

9 Optimal filters depend on SNR compared with contrast sensitivity functions (human) [Atick 92] Limitations Signal model: Gaussian, 1/f Noise model(s): Gaussian, uncorrelated Photoreceptors assumed regularly spaced Transformation assumed linear Transformation assumed convolutional

10 Cone:RGC ratio is not 1:1 outside fovea Cone lattice is not regular RGC receptive fields are irregular a b c d e a d b e c Curcio et al 91 [Gauthier et al, 9] Efficient Coding in Retina ν δ s H + W + r image blur cone noise retinal response transform noise [Doi et. al., SfN 28] Measured W Macaque retina, 27 deg eccentricity full mosaics: ON/OFF Midget/ Parasol 145 RGCs, 665 cones [ratio = 4.6] receptive fields, as weights on cones, determined by STA W contains receptive fields N=69 [Gauthier et. al., 9]

11 Information Maimization (total power of responses is constrained) Solution: W = PΩQ T P orthonormal, unconstrained [Doi etal, SfN 8] Information Maimization (total power of responses is constrained) Solution: W = PΩQ T Q orthonormal, evecs of blurred signal is of blurred signal [Doi etal, SfN 8] Information Maimization (total power of responses is constrained) Solution: W = PΩQ T with different with different photoreceptor SNR neural SNR with different population size 4 2 db 1 Modulation 2 2 db 1 db db 1 db 1 db db Eigenvector inde [Doi etal, SfN 8]

12 Theoretical W Pielated image Cone mosaic RGCs Optimal!"Ω based on: 62 natural images [Doi et al. 23] Human optical blur at 3 deg ecc [Navarro etal 93] 1 db photoreceptor SNR 1 db ganglion cell SNR [Borst & Theunissen 99] [Doi et. al., SfN 28] Comparison to data 65.7% variance eplained (34.3% error) Measured W Measured RFs Wmea Weff ( Pfit) Solution manifold Weff ( P) Random manifold Wrnd( P) Efficient manifold Random manifold 16.6% variance (+/-.1%) eplained (83.4% error) [Doi et. al., SfN 28] Parasol Midget ON OFF ON OFF N=5 N=6 N=26 N=32 theory data [Doi et. al., SfN 28]

13 Redundancy Information [bits] Physiological Optimal Random (a) (b) (c) 8 54% 49% 16% 18% 6 OFF-M OFF-M 32% 27% 2% 19% 4 22% ON-M M 25% ON-M M 2 9% All 6% All OFF-P 21% OFF-P 17% P P 1% ON-P 8% ON-P 4% 46% 31% OFF-M ON-M OFF-P 27% ON-P 26% M 3% P 3% 68% All [Doi, et. al., unpublished] ICA on image blocks [Bell/Sejnowski 97; see also Olshausen/Field 96] [eample obtained with FastICA, Hyvarinen] Learned kernels share features of auditory nerve filters Auditory nerve filters from Carney, McDuffy, and Shekhter, 1999 Optimized kernels scale bar = 1 msec NIPS 27 Tutorial 7 Michael S. Lewicki! Carnegie Mellon - Smith & Lewicki, 26

14 For most filters, there s a matching auditory nerve fiber! Trouble in paradise Biology: Visual system uses a nonlinear cascade - Where s the retina? The LGN? - What about V1 nonlinearities? - What happens after V1? Statistics: Images don t obey ICA source model - The responses of ICA filters are highly dependent [Wegmann & Zetzsche 9; Simoncelli 97; Lyu & Simoncelli 9] Conditional densities [Simoncelli 97]

15 a Baboon Flowers White noise Cat Speech White noise b [Schwartz & Simoncelli, 1] C Other Neurons C Other Neurons Divisive normalization [Heeger 92] reduces dependencies Can optimize parameters of this model for efficiency in representing natural images... [Schwartz & Simoncelli, 1] contrastindependence of orientation tuning orientation tuning of mask suppression a b Mean firing rate Mean firing rate 4 2 Cell (Skottun et al., 1987) -5 5 Orientation 1 Cell (Bonds, 1989) Model Contrast: Orientation Model single grating mask grating -1 1 Grating Orientation -1 1 Grating Orientation Cell (Bonds, 1989) Model cross-orientation suppression c Mean firing rate 4 Mask contrast: Signal Contrast Signal Contrast

16 Masking a b Mean firing rate Mean firing rate Cell (Cavanaugh et al., 2) Signal contrast Signal contrast Model Signal contrast Signal contrast Mask Mask Signal Mask contrast: No mask.13.5 Signal Mask contrast: No mask.13.5 c d Mean firing rate Mean firing rate Cell (Javel et al., 1978) Model Signal intensity (db) Signal intensity (db) Signal intensity (db) Signal intensity (db) Signal Mask Mask intensity: No mask 8 db Signal Mask Mask intensity: No mask 8 db Dependence of some tuning curves on input intensity Cell (Cavanaugh et al., 2) Model Cell (Rose et al., 1971) Model a Mean firing Rate Diameter (deg.) 3 6 Diameter (deg.) Contrast:.25.6 b Mean firing rate Rel. Freq. Decibels Rel. Freq.