The Puzzle of Perception

Transcription

1 The Puzzle of Perception Bruno A. Olshausen Helen Wills Neuroscience Institute, School of Optometry and Redwood Center for Theoretical Neuroscience UC Berkeley

2 Redwood Center for Theoretical Neuroscience UC Berkeley

3 IQ Engines Image and video search Biologically inspired approach Expertise in computational vision, neuroscience Start-up, UC Berkeley and UC Davis Dr. Bruno Olshausen UC Berkeley Dr. Fritz Sommer UC Berkeley Dr. David Warland UC Davis Pierre Garrigues UC Berkeley Gerry Pesavento Dr. Kilian Koepsell UC Berkeley Jack Culpepper UC Berkeley Charles Cadieu UC Berkeley

4 Main points The puzzle of perception Cybernetics and artificial intelligence What we know (and don t know) from neuroscience Where do we go from here?

5 sensors actuators neurons environment

6 What have brain scans and single-unit recording taught us about perception?

7 After 50 years of concerted research efforts... machines are still incapable of solving simple perceptual tasks. there is little understanding of how neurons interact to process sensory information. We are missing something fundamental on both fronts: we are ignorant of the underlying principles governing perception.

8 The Unknown As we know, There are known knowns. There are things we know we know. We also know There are known unknowns. That is to say We know there are some things We do not know. But there are also unknown unknowns, The ones we don't know We don't know. Feb. 12, 2002, Department of Defense news briefing From: The Poetry of Donald Rumsfeld Hart Seeley, Slate Magazine

9 Cybernetics and Artificial Intelligence

10 Artificial Intelligence Alan Turing John von Neumann Marvin Minsky John McCarthy Among the most challenging scientific questions of our time are the corresponding analytic and synthetic problems: How does the brain function? Can we design a machine which will simulate a brain? -- Automata Studies, 1956

11 Cybernetics/neural networks Norbert Wiener Warren McCulloch & Walter Pitts Frank Rosenblatt The theory reported here clearly demonstrates the feasibility and fruitfulness of a quantitative statistical approach to the organization of cognitive systems. By the study of systems such as the perceptron, it is hoped that those fundamental laws of organization which are common to all information handling systems, machines and men included, may eventually be understood. -- Frank Rosenblatt The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. In, Psychological Review, Vol. 65, No. 6, pp , November, 1958.

12 Perceptron model of a neuron x 1 x 2 x 3 x n... w 1 w 2 w 3 Σ w n w 0 u σ y o x 2 x x x o o o o o o o x x o x x x w x 1

13 NetTalk (Sejnowski & Rosenberg 1987)

14 LeNet (Yann LeCun et al., 1989)

15 Caltech 101

16 Superposition of all objects by category

17

18 DARPA Robots vs. Dung Beetles

19 What we know (and don t know) from neuroscience

20

21

22 Fly H1 neuron - dynamic range of speed sensitivity Lewen, Bialek & de Ruyter van Steveninck (2001) average rate (spikes/s) outside laboratory velocity ( /s)

23 Jumping spiders

24 Jumping spider visual system retinal photoreceptor mosaic

25 Jumping spiders do object recognition

26 Spider mimicry in flies

27 Human retina - cone mosaic

28 Fixational eye movements

29 Human fixational eye movements (Austin Roorda, UCB)

30

31 light

32 HI horizontal cell

33 HI horizontal cells connected via gap junctions

34 Lateral inhibition: activation of one photoreceptor inhibits neighboring photoreceptors

35 Bipolar cells read out differences between one photoreceptor s activity and its neighbors as computed by horizontal cell network - -

36 light

37 Visual information processing pathways Occipital lobe MT Posterior Parietal complex V1 V2 V4 Inferotemporal complex cortex Retina LGN Pulvinar thalamus (from visual cortex) Superior Colliculus midbrain

38 where Hippocampus.. what..?..... V1

39 Are there principles? God is a hacker Francis Crick...their (neurons ) apparently erratic behavior was caused by our ignorance, not the neuron s incompetence. H.B. Barlow (1972)

40 Principles of optics govern the design of eyes

41 Computational theories The efficient coding hypothesis Causal inference in probabilistic models

42 The efficient coding hypothesis ( Redundancy Reduction : Barlow 1961; Attneave 1954) Nervous systems should exploit the statistical dependencies contained in sensory signals to make efficient use of neural resources.

43

44 Neighboring pixels in natural images exhibit strong correlations 1 pixel separation 2 pixel separation 4 pixel separation I(x+1,y) I(x+2,y) I(x+4,y) I(x,y) I(x,y) I(x,y)

45

46

47

48

49

50 Visual information processing pathways Occipital lobe MT Posterior Parietal complex V1 V2 V4 Inferotemporal complex cortex Retina LGN Pulvinar thalamus (from visual cortex) Superior Colliculus midbrain

51 canononical microcircuit

52 Perception as inference lens World Image Model

53 Natural scenes are filled with ambiguity 2 Image cross section Pixel value Space (pixels)

54 Mooney faces

55 Mooney faces

56 Factorization

57

58

59 Object recognition depends on scene context

60 Object recognition depends on scene context

61 Object recognition depends on scene context

62 Sinewave speech

63 Sinewave speech Please say what this word is sill shook rust wed pass lark jaw coop beak

64 Bayesian inference

65 V1 receptive fields

66 Sparse, distributed representations a i I(x,y) image neural activities (sparse) features other stuff

67 Learned features (200, 12x12 pixels)

68 Hierarchical models... V2 V1 LGN

69 Shape representation in human visual cortex (fmri) (Scott Murray - Ph.D. thesis)

70 Moving diamond behind occluders

71 Moving diamond behind occluders (easy version)

72 BOLD signal: LOC vs. V : 012,3&4"'5!%&'(#"%)!"#"$%&'(#"%)!"#"$%&'(#"%)!%&'(#"%) *&(+,-./ *&(+,-./

73 What is the other 85% doing? 1.0 Biased sampling (single unit recording) Variance explained ~0.4 ~85% of V1 function not understood Biased stimuli (bars, spots, gratings) Biased theories (data-driven vs. functional theories) Interdependence and context (effect of intra-cortical inputs) Proportion of cells studied Ecological deviance Olshausen BA, Field DJ (2005) How close are we to understanding V1? Neural Computation, 17,

74

75

76

77

78

79

80 Anatomy of a synapse

81 Where do we go from here? Computer scientists and neuroscientists need to recognize they are confronting the same problem. Exploratory studies vs. experiments designed to test specific hypotheses. Better technology - need to view simultaneous activity of many neurons. Humility

82 Silicon polytrodes

83

84 our ability to reverse engineer the brain - to see inside, model it, and simulate its regions - is growing exponentially. We will ultimately understand the principles of operation underlying the full range of our own thinking, knowledge that will provide us with powerful procedures for developing the software of intelligent machines.... There are no inherent barriers to our being able to reverse engineer the operating principles of human intelligence and replicate these capabilities in the more powerful computational substrates that will become available in the decades ahead. The human brain is a complex hierarchy of complex systems, but it does not represent a level of complexity beyond what we are already capable of handling.