2012 Course : The Statistician Brain: the Bayesian Revolution in Cognitive Science

Transcription

1 2012 Course : The Statistician Brain: the Bayesian Revolution in Cognitive Science Stanislas Dehaene Chair in Experimental Cognitive Psychology Lecture No. 4 Constraints combination and selection of a unique percept Lecture material translated from the French version by CG Traduction et Interprétation

2 sensation S Why does the Bayesian framework fit perception so well? objet O Sensorial inputs are always ambiguous. Perception of size: horizontal/vertical illusion Our perceptual system must therefore select the most plausible solution among an infinite number of possibilities. Perception of orientation: a priori for horizontal and vertical Pictures from the ScholarPedia article «Visual_illusions:_An_Empirical_Explanation» by Dale Purves et coll.

3 Example 3: perception of movement and the aperture problem

4 The aperture problem All local movement, perceived through a narrow field, is compatible with an infinite number of possible interpretations Speed in y Vertical velocity (deg/s) Horizontal velocity (deg/s) Speed in x

5 The aperture problem When a surface is in motion, global movement can be calculated on the basis of the combination of local speeds. Speed in y Vertical velocity (deg/s) Horizontal velocity (deg/s) Speed in x

6 Several mechanisms were suggested: intersection of constraints vector average, VA feature tracking, FT.

7 VA, IOC and FT mechanisms alone can not explain perception of movement

8 The perception of movement can be seen as a Bayesian problem: Which movement is the most probable in view of the available cues? The model postulates that likelihood functions are contrast-dependent: Plus a prior centered on zero (the absence of movement is the most likely).

9 The model can explain the diamond-shape illusion:

10 The model covers a large number of different illusions: Speed of a grating direction of a «plaid» direction of a curve direction of a «plaid»

11 The model can also explain other illusions: - why do we perceive that our movement is slower when driving in the fog? - why do we sometimes perceive two overlapping gratings as moving independently instead of seeing them as one single «plaid»? For more information on the many regularities that are internalized through visual perception, see: Kersten, D., Mamassian, P., & Yuille, A. (2004). Object perception as Bayesian inference. Annu Rev Psychol, 55,

12 Perceptual inference can be much more complex than in the previous example Sensorial inputs are frequently the result of a complex combination of variables and sources of noise. Constraint propagation can be modeled via Bayesian networks. Depending on the task at hand, the relative importance of the parameters has to be assessed. Kersten, Mamassian et Yuille, Annual Review 2004

13 Example: perception of light and its influence on the perception of concave and convex shapes Without specific guidance, the visual system «hypothesizes» that the light comes from above (and from the left) Ramachandran, V. S. (1988). Perceiving shape from shading. Sci Am, 259(2), The visual system efficiently integrates priors with perceptive cues to revise its judgment. Morgenstern, Y., Murray, R. F., & Harris, L. R. (2011). The human visual system's assumption that light comes from above is weak. Proc Natl Acad Sci U S A, 108(30),

14 Bayesian integration of sensory information Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), Let s imagine that we simultaneously receive visual and haptic cues on the size of an object. How do we combine visual and haptic information?

15 Bayesian integration of sensory information Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), If these cues are conditionally independent, Bayes rule states that their combined probability density is the product of both densities. (, ) P wtv = = (, ) ( ) Ptv (, ) ( ) ( ) ( ) Ptv (, ) ( ) ( ) ( ) Ptv wp w P t w P v w P w P t w P v w P w

16 Bayesian integration of sensory information Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), The product of two Gaussian functions is a new Gaussian function According to the principle of maximum likelihood: -Perception is the weighted average of the values suggested by each cue -Weights depend on the reliability of cues (as opposed to variances) -The total reliability is the sum of reliabilities (Fischer s information is additive for independent signals)

17 Bayesian integration of sensory information Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), Findings: -The perceived size shifts in the direction of the size suggested by vision -In direct proportion to the reliability of visual cues -With a decreasing noise level as reliability increases

18 Bayesian integration of sensory information Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), The accuracy of the response (JND = just noticeable difference) is consistent in quantitative terms with the predictions of Bayes theory Conclusion: The perceptual system integrates sensations derived from two sensorial modalities according to the rules of Bayesian inference.

19 Ambiguity and conscious awareness: Very often, we are only able to perceive a sample of the underlying probability distribution

20 The perception of ambiguous figures is an example of Bayesian sampling Moreno-Bote, R., Knill, D. C., & Pouget, A. (2011). Bayesian sampling in visual perception. Proc Natl Acad Sci U S A, 108(30), Creation of an ambiguous stimulus with two possible interpretations (two overlapping gratings). We subjectively perceive alternations of A in front of B and A behind B. The dominance of one percept over the other can be manipulated with visual cues: wavelength and speed (exp. 1) or wavelength and binocular disparity (exp. 2) The Bayesian model predicts that if cues are conditionally independent, perception should follow the multiplicative rule:

21 The perception of ambiguous figures is an example of Bayesian sampling Moreno-Bote, R., Knill, D. C., & Pouget, A. (2011). Bayesian sampling in visual perception. Proc Natl Acad Sci U S A, 108(30), Findings: Whether cues are congruent or incongruent, the product rule can be used to predict the fraction of the perception of the items, on the basis of the distributions observed when only one cue is manipulated. The correspondence with experimental curves is close to perfect, contrary to other models, such as for example when supposing that there is only one dominant cue. Conclusion: the fraction of dominance in bistable perception behaves as a probability

22 How to model Bayesian sampling? Moreno-Bote, R., Knill, D. C., & Pouget, A. (2011). Bayesian sampling in visual perception. Proc Natl Acad Sci U S A, 108(30), The data can be explained by a two-attractor model. Input I redirects the energy function towards attractor A or B. The system can be switched from one attractor to the other by the use of noise. (B). The system swings between two states (C) The fraction of time spent in each state follows the product rule (E), only if the input is a sum of inputs I 1 et I 2. Why? The fraction of time spent in one state is a sigmoïd function, therefore close to an exponential exp(i/σ²) If the input is a linear function of two cues I = I 1 + I 2, then this model has the multiplicative property precisely required, because: exp((i 1 + I 2 )/σ²) = exp(i 1 /σ²) exp(i 2 /σ²)

23 How to model Bayesian sampling? Moreno-Bote, R., Knill, D. C., & Pouget, A. (2011). Bayesian sampling in visual perception. Proc Natl Acad Sci U S A, 108(30), This theoretical model can be simulated by a formal neural network (and can be implemented with realistic neural firing; see also Wong et Wang, J Neuroscience 2006). The simulation yields new important properties: - the life cycle of attractors follows a gamma distribution (in red, predictions of the model; in blue, empirical data) - when the stimulus parameters are changed, the changes induce a systematic relationship between the mean duration and the dominance fraction:

24 The conscious sampling hypothesis Vul, E., Hanus, D., & Kanwisher, N. (2009). Attention as inference: selection is probabilistic; responses are all-or-none samples. J Exp Psychol Gen, 138(4), According to Vul et al., all perceptive and attentional computations are probabilistic. However, at the time of response, the resulting distribution of probability may be sampled by the brain. Sampling may be a response to the difficulty of working out a complete Bayesian computation with the full distribution pattern. Vul s PhD thesis demonstrates that computation based on only a small number of samples suffices to come to the right decision and that in cases where time pressure is applied, the use of one single sample optimizes the average reward per time unit. Testing the model: Selecting and reporting a letter.

25 The proposed Bayesian model: The conscious sampling hypothesis Vul, E., Hanus, D., & Kanwisher, N. (2009). Attention as inference: selection is probabilistic; responses are all-or-none samples. J Exp Psychol Gen, 138(4), According to Vul, the subject s answers sample this distribution (instead of selecting the full span)

26 The conscious sampling hypothesis Vul, E., Hanus, D., & Kanwisher, N. (2009). Attention as inference: selection is probabilistic; responses are all-or-none samples. J Exp Psychol Gen, 138(4), The fact that errors across trials are spread around the real target, is not sufficient to establish the veracity of this model. In fact, the variability of responses may be caused by inter-trial variance in the selection of the target letter. The attentional selection is supposed to be all-ornone, but its point of application could vary across trials. How can this interpretation be refuted?

27 The conscious sampling hypothesis Vul, E., Hanus, D., & Kanwisher, N. (2009). Attention as inference: selection is probabilistic; responses are all-or-none samples. J Exp Psychol Gen, 138(4), Solution = asking for several answers in each trial: Neither the second not the third answer are random, and their frequencies remain unchanged:

28 The conscious sampling hypothesis Vul, E., Hanus, D., & Kanwisher, N. (2009). Attention as inference: selection is probabilistic; responses are all-or-none samples. J Exp Psychol Gen, 138(4), Answer 2 acts as a second sample of the same distribution, completely independent from the position of answer 1.

29 The conscious sampling hypothesis Vul, E., Hanus, D., & Kanwisher, N. (2009). Attention as inference: selection is probabilistic; responses are all-or-none samples. J Exp Psychol Gen, 138(4), Results are replicated in a second experiment where selection is spatial rather than temporal.

30 The conscious sampling hypothesis Vul, E., Hanus, D., & Kanwisher, N. (2009). Attention as inference: selection is probabilistic; responses are all-or-none samples. J Exp Psychol Gen, 138(4), Conclusions: - The findings show that the process of attentional temporal and spatial selection operates gradually and statistically - Answers vary between trials but this entire variance seems to be the result of the intratrial variance in the selection function - The reported answers act as samples of an internal distribution that obeys the rules of Bayesian inference [although this last point has not yet been very systematically tested by Vul et al.]. - Although brain functions are probabilistic, we are only aware of discrete samples: - Selective attention works at a non-conscious and continuous level - Access to consciousness works on an all-or-none basis - These findings are compatible with those of Sergent and Dehaene (2004, 2005)

31 Is there an internal sampling process during cognitive inference? Vul, E., & Pashler, H. (2008). Measuring the Crowd Within: Probabilistic Representations Within Individuals. Psychological Science (Wiley-Blackwell), 19(7), «The Wisdom of Crowds»: Galton (1907) shows that, when people are asked to appraise the weight of an ox, the error of the group average is lesser than the average error of each respondent. It makes sense to ask «the crowd for its opinion» In the experiment, 428 subjects respond to questions such as «what percentage of the world s airports are in the United States?» Half of the trial population was asked for a second guess immediately, whereas the other half was asked 3 weeks later. Surprisingly, when the question is asked twice, the average of the two answers is better than either the first or the second answer:

32 Is there an internal sampling process during cognitive inference? Vul, E., & Pashler, H. (2008). Measuring the Crowd Within: Probabilistic Representations Within Individuals. Psychological Science (Wiley-Blackwell), 19(7), The degree of improvement can be compared to that obtained when the question is asked to another person. The result is 1/10 for an immediate second guess, and 1/3 for a second guess three weeks later. Conclusion: even plausible cognitive inferences, formulated from memory, may rely on a form of probability distribution sampling.

33 Conclusion: inference and sampling Whether we perceive a stimulus or make a decision on the basis of the cues at our disposal, the perceived or selected value acts as a sample of a posteriori distribution. According to Bayesian inference rules, this distribution combines the priors of our initial knowledge of the world and the indications or sensations we collect at a given moment in time. Why sample a distribution? Sampling is not necessarily the best solution: it may be more appropriate to work with the full distribution or to always select the most probable solution (maximum a posteriori, MAP). Several possibilities: - Beyond a given level of complexity, computation is only possible via sampling. - The nervous system can represent distributions, but can only make decisions on the basis of single samples. - In a changing world, sampling enables us to explore and remain open to change (dilemma between exploitation and exploration). - Noise may also be useful in finding the optimum (simulated annealing).