Lecture. Experimental Models of Introspection in the Animal

Introspection and metacognition: Mechanisms of self-knowledge Stanislas Dehaene Chair of Experimental Cognitive Psychology Lecture Experimental Models of Introspection in the Animal Course translated from the French version by CG Traduction et Interprétation

How I love the scent of tanning lotion early in the morning! The New Yorker, 2006

Are animals endowed with a form of introspection? Many experiments on metacognition are based on verbal or digital reports (degree of confidence in the answer). Metacognitive skills are not typically y observed before the age of 4, a fact which has led some to believe that metacognition requires language. (Metcalfe & Shimamura, 1994; Tulving, 1994). However, as covered in the preceding lecture, non-verbal methods have shown that the theory of mind can be attributed to infants aged 14 and sometimes even 7 months. Kornell, Son & Terrace (2007) propose two testing methods for metacognition without language : 1. Introspection (metacognitive monitoring): Can the animal report on its own degree of confidence in its answers, for example by choosing the level of risk or non-risk in its responses? 2. Metacognitive control: can the animal show that it «does not know» through active research for more information, for example?

Smith s Paradigm: the «uncertain» response Smith, J. D., Shields, W. E., Schull, J., & Washburn, D. A. (1997). The uncertain response in humans and animals. Cognition, 62(1), 75-97. Judgment of the density of screen The «star» key provides escape from the primary task and prompts a modest but invariant reward. The animal can then manifest its «uncertainty» or its «lack of self-confidence» macaques humans

Smith s Paradigm in the bottlenosed Dolphin Smith, ih J. D., Schull, J., Strote, J., McGee, K., Egnor, R., & Erb, L. (1995). The uncertain response in the bottlenosed dolphin (Tursiops truncatus). J Exp Psychol Gen, 124(4), 391-408. Judgment of pitch height: 2100 Hz or a lower frequency? The animal can refuse the test and is then assigned an easier one. Discrimination is excellent, but the animal specifically refuses difficult tests. The animal can also show signs of hesitation (swims slowly, shakes its head!), precisely in the same frequency range.

Terrance and Son s Critical Classification (2009)

Monkeys know when they remember Hampton, R. R. (2001). Rhesus monkeys know when they remember. Proc Natl Acad Sci U S A, 98(9), 5359-5362. 5362 Prospective Metacognition (before completing the Lpr imary task): The animal must memorize an image. In some cases, a memory test is imposed (left). In others, the animal may choose to refuse and is given a small but invariant reward. In two cases, monkeys perform memory tests better when they are free to choose to take the test (black bars) rather than when the test is mandatory (grey bars)

Monkeys know when they remember Hampton, R. R. (2001). Rhesus monkeys know when they remember. Proc Natl Acad Sci U S A, 98(9), 5359-5362. 5362 Experimental Control (Experiment 2): Experiment 3: In a fraction of the tests, no image is presented Memory Delay Variations: during the memorization phase. If the monkeys Animals refuse the test more and more understand the task, they must always refuse frequently when the time interval is the memory test. If, on the other hand, their increased. Their performance is better performance is based on other behavioral cues when they are given the option to refuse. (movement, auditive distraction), then their behavior should remain constant regardless of whether or not images are presented. Result: monkeys refuse the tasks much more often when there are no images to memorize (grey bar).

An authentic demonstration of metacognition in the animal Kornell, Son & Terrace, Psychological Science 2007 Retrospective Metacognition: after performing the task, but prior to receiving the reward, the animal must judge its own performance. Two new criteria for an authentic demonstration of metacognition in the animal Criterion 1: the animal does not only need to learn to use the avoidance response at the right time, but must also be able to generalize this response to new tasks Generalization tends to demonstrate that the animal effectively chooses the invariant reinforcement response to manifest its degree of confidence in its answer. And that t this response is different from responses evoked through h a restricted type of stimuli. In Kornell et al. s (2007) experiment 1, two macaque monkeys 1. learn to use the invariant reinforcement response in a size-assessment task 2. immediately generalize this response to a new memory task.

An authentic demonstration of metacognition in the animal Primary task = Size-assessment: choosing the biggest disk. Secondary task = risk-taking The animal learns that 1. if it makes the correct choice, and clicks lik on the appropriate it icon, it is rewarded with 3 kibbles. 2. if it makes an incorrect choice and presses on the right-hand icon, 3 kibbles are withdrawn. 3. if it presses on the left-hand icon, regardless of its choice, it is rewarded with a single kibble. It is to be noted that the metacognitive response is given after the removal of the primary task stimuli. Kornell, Son & Terrace, Psychological Science 2007

An authentic demonstration of metacognition in the animal Kornell, Son & Terrace, Psychological Science 2007 Primary Task N 2 = Shortterm memory : After viewing a series of 6 images, the animal must recognize in the subsequent series of 9 images, the one which also appeared in the preceding series. In a first stage, the animal must learn the task. Then, the judgment of risk- taking is added (over 20 seconds after the stimuli).

An authentic demonstration of metacognition in the animal Kornell, Son & Terrace, Psychological Science 2007 Results are quantified through the correlation (phi) between the successful outcome of the primary task and risk-taking. tki Observations reveal significant correlation in the two animals tested: -nearly perfect generalization to the other primary task (memory). Do the animals use elementary cues? No: - correlation is maintained regardless of the position of the target in the series list; the judgment of the animal is therefore not based on this cue - metacognitive performance is negatively correlated to response time (mistakes are lower), but remains significant even when the effect of the response time is subtracted by regression.

An authentic demonstration of metacognition in the animal Kornell, Son & Terrace, Psychological Science 2007 Criterion 2 = capacity for metacognitive control : The animal must be able to use its degree of confidence to actively seek new information. In preceding experiments (Call & Carpenter, 2001; Hampton, Zivin, i & Murray, 2004), if animals saw where food was hidden, they immediately went to the appropriate hiding spot; if they did not see, they actively searched for the food. For Terrace, this does not mean that they «know that they do not know». Retrieval and search for food are spontaneous behaviors, triggered directly by the knowledge of where the food is (or not knowing), which is not metacognition («knowing that we know»). In Experiment 2 animals learn a new sequence of images, through trial and error, with a cue given by an additional key. Results show that they use this key appropriately.

An authentic demonstration of metacognition in the animal Kornell, Son & Terrace, Psychological Science 2007 Task= Discover the order in which a series of images must be placed (by touching a tactile screen). The additional «cue» key is available in 50% of the tests. A click on this key tells the animal which element comes next in the sequence. This also modifies the reward: -the red dot indicates that if successful, the animal will be rewarded with one M&M - clicking on the «cue» key erases the red dot: the reward will then only be a banana-flavored kibble.

An authentic demonstration of metacognition in the animal Results: The proportion of clicks on the «cue» key diminishes in correlation with the memorization of the lists. This negative correlation applies to each list, whether familiar or new, but the number of requests for cues increases considerably with the introduction of a new list. Kornell, Son & Terrace, Psychological Science 2007 + This is the very first experiment in the study of the animal s active search for information - This experiment is slightly less convincing than the first: - Does the animal really understand that the «cue» means that additional information is available? A generalization experiment would have been useful to answer that question. Is it really impossible to interpret this experiment in terms of reward maximization? In the case of a new list, the use of the cue key brings faster reward; when lists are familiar, the reward is maximized if the cue key is not used. It could very well be that the animal uses elapsed time as an indication of the «novelty» of the list and that the variable «I know that I know this list» does not enter into play in the decision. i Correlation is not causation!

Towards the neurophysiology of introspection The metacognitive protocols which evaluate the degree of an animal s self-confidence are now simple enough to allow for neurophysiological experimentation. Two very recent articles on the topic : Kiani, R., & Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. Science, 324(5928), 759-764. Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008). Neural correlates, computation and behavioural impact of decision confidence. Nature, 455(7210), 227-231.

Relevant decision-making statistics are compiled through the prefrontal and parietal neurons The monkey must decide on the main direction of a movement. The consistency of the movement modulates the increase in neuronal responses during the decision-making process. Decision Fixed threshold Variable speed accumulation Kim & Shadlen, Nature Neuroscience 1999

Neural signals associated with decision-making processes also encode information on the relevant degree of confidence. Neural responses recorded in particular during decision-making processes differentiate between correct and erroneous tests. The difference in neuronal activity encoding both possible decisions could give an indication as to the degree of confidence in the decision. (Mechanism proposed by Vickers, 1979) Kim & Shadlen, Nature Neuroscience 1999

Representation of confidence by neurons in the parietal cortex Kiani, i R., & Shadlen, hdl M. N. (2009). Representation of confidence associated with iha decision ii by neurons in the parietal cortex. Science, 324(5928), 759-764. Conventional task of Newsome & Shadlen: - decision on the general direction of the movement of a cloud of random dots New ingredient : - prior to the decision, a third target, called sure target, is added in some trials - if this target is selected, the reward is invariant but modest. - it is possible to interpret this choice as a refusal to respond to the main task, which could be an indication of «uncertainty» or «lack of self-confidence» in the animal.

Behaviors appropriately reflect the representation of the animal s degree of confidence in its own judgment Kiani, R., & Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. Science, 324(5928), 759-764. The monkey refuses to respond all the more that the probability of a correct response to the primary task is low. - The estimation of uncertainty (type II response, left) and objective performance (type I response, right) are mirror symmetric graphs. -More importantly, objective performance is found to be better when the animal has the option to refuse (continuous line) vs when this option is not available (dotted line). - This points to the fact that, under identical stimuli, the animal systematically rejects tests where it (rightly) thinks that it is unable to give the correct answer.

Neuronal recordings are modulated by the animal s confidence Kiani, R., & Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. Science, 324(5928), 759-764. Option to refuse unavailable option to refuse available : Continuous line = option waived (risk-taking) Dotted line = option adopted (lack of confidence) The firing rate in neurons where receptors include one of the primary responses reflects - the future decision i (left graph) - as well as the confidence in the response (right graph) - and this, even before the animal knows whether the default option will be proposed or not. Important control: when the animal chooses to refuse to respond, variance in neuronal firing rates is low, which rejects the hypothesis of mixed high and low states.

Neuronal recordings are modulated by the animal s confidence Kiani, R., & Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. Science, 324(5928), 759-764. Trial-to-trial variation of neuronal firing rates, as measured immediately before the default option is proposed, predicts the choice of this option. - this is also found in the analysis of trials with strictly tl identical stimulus conditions. The speed of firing buildup during stimulus appearance also makes a contribution independently of the choice of the default option.

Theoretical confidence model Kiani, R., & Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. Science, 324(5928), 759-764. 764 The decision variable fluctuates over time, thus fairly strongly predicting success or failure. Assumption: the animal applies a second-order criterion to this internal variable as measured at the decision-making moment. Result: This simple assumption generalizes the theoretical model of second-order signal detection and suffices to model behavior and neuronal discharges

Neuronal activity associated with second-order decisions Kiani, R., & Shadlen, M. N. (2009). Representation of confidence associated with a decision by neurons in the parietal cortex. Science, 324(5928), 759-764. - When the receptor field of the neuron coincides with the position of option II, we can observe evolutions in the confidence judgment. Neuronal discharge begins after the presentation of option II. This is indeed a metacognitive response rather than a judgment based on the choice between three alternatives. Conclusion: the monkey is endowed with an efficient decision assessment mechanism (not necessarily located in the parietal cortex)

Neuronal representation of confidence in the rat Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008). Neural correlates, computation and behavioural impact of decision confidence. Nature, 455(7210), 227-231. Discrimination between two odors, presented as a mixture of variable proportions. Reward occurs following a variable time interval of 0,3 to 2 seconds. Neuronal recordings in the orbitofrontal cortex, region frequently associated with reward anticipation.

Neuronal representation of confidence in the rat Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008). Neural correlates, computation and behavioural impact of decision confidence. Nature, 455(7210), 227-231. -Neuronal responses cannot be explained by the association between stimuli and reward (despite the recent history of such associations). - On the other hand, they are clearly explained by the type II signal detection theory, where the degree of confidence is determined by the distance between the decision variable and the type I decision criterion. erreurs réponses correctes -Rats have the capacity to use this information ion: When given the opportunity to repeat a test by returning to the central point, without waiting for the reward, rats do it all the more that they previously gave incorrect answers, showing a very similar curve in neuronal response. Thi ld i d i i i b b d h l i f This could point to reward anticipation, but based on the analysis of the quality of the decision rather than only on the stimulus presented authentic metacognition in the rat

Conclusions The uncertainty estimate seems to form an integral part of the decision. It is found in many animal species. Even more impressive is the fact that these animals are able to use their uncertainty estimates to modify their behavior. This is indeed a second-order or metacognitive judgment, that is not necessarily conscious). However, laboratory experiments pose the issue of intensive training of animals An ethological analysis should be undertaken to check whether these mechanisms are used in the animal s natural environment. Gary Larson