Primate polemic: Commentary on Smith, Couchman and Beran (2013) 1 Mike E. Le Pelley AUTHOR S MANUSCRIPT COPY This is the author s version of a work that was accepted for publication in the Journal of Comparative Psychology. Changes resulting from the publishing process, such as editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published as: Le Pelley, M. E. (2014). Primate polemic: Commentary on Smith, Couchman and Beran (2013). Journal of Comparative Psychology, 128, 132-134, doi: 10.1037/a0034227.
2 Primate polemic: Commentary on Smith, Couchman and Beran (2013) Mike E. Le Pelley School of Psychology, University of New South Wales, Sydney, Australia Please address correspondence to Dr Mike Le Pelley School of Psychology University of New South Wales Sydney NSW 2052 Australia Tel: +61 2 9385 1294 Email: m.lepelley@unsw.edu.au
Abstract 3 Smith, Couchman and Beran (2013) take issue with recent attempts to account for socalled metacognitive behaviour in non-human animals in terms of simple processes of associative reinforcement learning. Their arguments rely on appeals to unconvincing and equivocal empirical evidence, and a misrepresentation of the nature of associative learning. While the existing data do not rule out the possibility that animals possess true metacognitive abilities, neither do they currently mandate this conclusion. The suggestion that simple mechanisms might give rise to complex behaviours ties in with recent attempts in cognitive and social psychology, and behavioural neuroscience, to explain human behaviour in terms of similar, simple mechanisms. As such this perspective should be seen as an opportunity for comparative psychology, not a threat. KEYWORDS: metacognition, uncertainty, associative learning, reinforcement learning, conditioning
A growing body of literature claims that animals share with humans the ability to 4 metacognitively monitor their internal states of uncertainty. Recently, however, researchers have proposed a re-evaluation of claims of animal metacognition in terms of simpler processes of associative reinforcement learning (Jozefowiez, Staddon & Cerutti, 2009; Le Pelley, 2012), arguing that studies of so-called metacognitive behaviour in non-human animals do not require us to impute metacognitive abilities to those animals at all. Smith, Couchman and Beran (2013) have attacked this killjoy approach (cf. Shettleworth, 2010) to so-called metacognition, in terms of the specific accounts offered and the general thesis they represent. However, these criticisms are ill-founded. Simulating metacognitive behaviour In their attack on associative accounts of so-called metacognitive behaviour, Smith et al. (2013) reference the recent claim that deferring feedback from a set of trials can rule out associative processes in driving behaviour in metacognition tasks (Couchman, Coutinho, Beran & Smith, 2010). This claim is untrue. It is easy to see that while deferring feedback weakens the relationship between behaviour and reinforcement, it does not dissociate them since reinforcement remains entirely determined by behaviour (see Le Pelley, 2012). Rather than attempt to dispute this conclusion, Smith et al. choose to ignore it, simply restating that deferring feedback means the processes of conditioning and association are ruled out (p4, p8). They proceed to detail the application of the associative model that I described (Le Pelley, 2012) to Smith et al. s (2006) asymmetry finding. Smith and colleagues have come to place great weight on this finding, stating here that the stakes are high for associative models to try to explain [it]. I disagree. As previously noted (Le Pelley, 2012), it is highly likely that this asymmetry was an artefact. Briefly, (a) it was demonstrated in one task by one monkey, (b) with no statistical analysis, (c) after
5 extensive previous training which would itself induce asymmetry, and (d) in a situation wherein the monkey s perception of stimuli may be asymmetrical. The sensible conclusion is that there is nothing for associative models to try to explain. This was pointed out to Smith et al. (2013) several times during review of their article; each time they did not dispute or address these inconvenient possibilities, but chose to ignore them. Until genuine asymmetry of uncertainty responding is demonstrated reliably and replicably, it is an inappropriate yardstick against which to judge theories. Parameters and phylogenetics Smith et al. (2013) further criticize associative accounts for their number of free parameters, in contrast with an account in which animals are only granted a basic capacity to monitor their memories (p24). But this criticism glosses over a crucial difference between the two approaches. The associative account invokes more parameters because it is an explanatory theory. It begins as a blank slate, and explains exactly how animals develop representations, and why they choose responses, based on experience. Smith et al. s metacognitive account does not. It simply assumes that animals have beliefs about relationships between stimuli and responses with no explanation of how these beliefs arose. It then assumes a homunculus adjudicating between beliefs, with no explanation of what that involves at a process level. This approach has fewer parameters because it is simply a set of vague principles, not a viable model; as Jozefowiez et al. (2009) put it, no process is proposed, no computable theory offered (p37). And certainly, if the metacognitive approach were to be fleshed out so that it actually explained the processes underlying behaviour, it would be far from basic. Expanding on this point, the goal of the associative approach is not to develop a specific model of uncertainty responding. Instead the aim of this modelling is to
demonstrate that so-called metacognitive behaviour drops naturally out of a simple, 6 general-purpose model of associative reinforcement learning. So the model s parameters are not there to explain metacognitive behaviour; they are invoked because they are needed by any general-purpose model of associative learning, and should be viewed in that light. Even Smith et al. would presumably concede that reinforcement learning occurs, and hence their approach must impute all the parameters needed by a general-purpose learning system in addition to special parameters required by their proposed metacognitive system. Given their criticism of the free parameters invoked by associative models, it is also peculiar that Smith et al. make no comment on how the predictions of these models change when these parameters are varied. For example, they note that capuchins are less inclined to make the uncertain response than macaques (Beran, Smith, Coutinho, Couchman & Boomer, 2009). This is taken to undermine an associative account of macaques uncertainty responding; if macaques metacognitive behaviour is associatively-based, then capuchins should behave similarly because they are equally capable of associative learning. But what the authors are ignoring is that an associative model can be parameterized such that it rarely, if ever, chooses the uncertain response. For example, in the associative model that I proposed (Le Pelley, 2012), decreasing the negative value of the timeout (valerr), or decreasing the positive value of the reward (valcorr), will reduce the likelihood of making the uncertain response. So it is easy to reconcile the behaviour of macaques and capuchins within an associative model, by choosing parameters appropriately for each species. And this produces specific, testable predictions. For example, perhaps capuchins are less sensitive to timeouts than macaques. If so, then increasing timeout duration should make capuchins more likely to make respond uncertain, and indeed lengthening the timeout did cause one capuchin to develop a metacognitive response pattern 1. Directly
reinforcing the third response to difficult stimuli (creating the so-called Middle 7 condition) also increased use of this response; another prediction of the associative model. The phylogenetic argument s weakness is further illustrated by data from studies with pigeons. Smith et al. note that Roberts et al. (2009) failed to find evidence for information-seeking by pigeons. Since pigeons are deft associative learners, they infer that information-seeking must have a non-associative basis. What they do not acknowledge (though it was pointed out during review) is that Zentall and Stagner (2010) identified a confound in Roberts et al. s procedure; when this was corrected pigeons showed clear evidence of information-seeking. Failure to observe a particular behaviour in a given species may be because that behaviour lies beyond the (meta)cognitive capability of the species, or because the experimental parameters and procedures are not optimized for revealing that behaviour in that species. Until the latter possibility is ruled out, it is premature to assume the former. What is associative learning? Smith et al. titled one section of their article Misunderstanding associative learning ; this title is apt. Their description of an associative approach applied to Hampton s (2001) study has the monkey searching memory locations, and making response decisions based on the search results. They argue that this account is indistinguishable from invoking true metacognition, so the term associative is vacuous. This would be true, if their caricature of associative learning weren t inaccurate. An animal is in state X; it performs response Y and is rewarded. When that animal finds itself in state X in future 2, it will other things being equal perform response Y. This is the essence of associative learning. It does not involve a search of memory, or deciding how to respond based on inspection of memory s contents. It is observed at
the level of individual neurons (Brembs, Lorenzetti, Reyes, Baxter & Byrne, 2002; 8 Malenka & Nicoll, 1999), or in artificial networks (Sutton & Barto, 1998), that have no notion of searches or decisions. And as such, it is very different from metacognition. I firmly believe that progress can be made through constructive discussion between associative theorists and researchers of higher-level cognition, by exploring the limits of associative models and designing good experiments to see if/when behaviour goes beyond those limits. However, the anti-associationist polemic of Smith et al. is unlikely to produce genuine progress. In their approach, an associationist account should be disfavoured because it carries risks to the development of comparative psychology (p7), and downgrades the relevance of animal research to human researchers (p32). This suggestion that comparative psychologists should be generally wary of associative accounts is ultimately regressive. Far from denying mental continuity between humans and non-humans, this work raises the possibility that seemingly complex human behaviour might also be understood as the product of relatively simple mechanisms. Consequently, greater appreciation of such mechanisms would contribute to a deeper, more truly comparative psychology (Shettleworth, 2010, p477).
Footnotes 9 1 Notably, this observation undermines Smith et al. s argument. They take adaptive uncertainty responding as evidence of metacognitive abilities. But if this capuchin had metacognitive abilities, why were they not used with shorter timeouts? It isn t clear why it would have been any less uncertain in that case. 2 An animal s state is influenced by its current perception and the residual influence of previous perceptions and behaviours. In Hampton s (2001) study, the state that associates with (and later cues) responses during the choice phase could be the residual activity of a representation of recently-experienced stimulus from the sample phase (as suggested by Le Pelley, 2012). Or, more generally, it could be strong residual activity of a representation of having recently seen a clip-art image. Or it could be activity from having recently looked at a screen (which will correlate with performance in the delayed-matching-to-sample test, since if the monkey did not look at the sample phase screen, it is unlikely to respond correctly).
References 10 Beran, M. J., Smith, J. D., Coutinho, M. V. C., Couchman, J. J., & Boomer, J. B. (2009). The psychological organization of "uncertainty" responses and "middle" responses: A dissociation in Capuchin monkeys (Cebus apella). Journal of Experimental Psychology: Animal Behavior Processes, 35, 371-381. doi: 10.1037/A0014626 Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., & Byrne, J. H. (2002). Operant reward learning in Aplysia: Neuronal correlates and mechanisms. Science, 296, 1706-1709. doi: 10.1126/science.1069434 Couchman, J. J., Coutinho, M. V. C., Beran, M. J., & Smith, J. D. (2010). Beyond stimulus cues and reinforcement signals: A new approach to animal metacognition. Journal of Comparative Psychology, 124, 356-368. doi: 10.1037/A0020129 Hampton, R. R. (2001). Rhesus monkeys know when they remember. Proceedings of the National Academy of Sciences of the United States of America, 98, 5359-5362. Jozefowiez, J., Staddon, J. E. R., & Cerutti, D. T. (2009). Metacognition in animals: How do we know that they know? Comparative Cognition & Behavior Reviews, 4, 29-39. Le Pelley, M. E. (2012). Metacognitive monkeys or associative animals? Simple reinforcement learning explains uncertainty in nonhuman animals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 686-708. doi: 10.1037/a0026478 Malenka, R. C., & Nicoll, R. A. (1999). Long-term potentiation: A decade of progress? Science, 285, 1870-1874.
11 Roberts, W. A., Feeney, M. C., McMillan, N., MacPherson, K., Musolino, E., & Petter, M. (2009). Do pigeons (Columba livia) study for a test? Journal of Experimental Psychology: Animal Behavior Processes, 35, 129-142. doi: 10.1037/A0013722 Shettleworth, S. J. (2010). Clever animals and killjoy explanations in comparative psychology. Trends in Cognitive Sciences, 14, 477-481. doi: 10.1016/j.tics.2010.07.002 Smith, J. D., Couchman, J. J., & Beran, M. J. (2013). Animal metacognition: A tale of two comparative psychologies. Journal of Comparative Psychology. Smith, J. D., Redford, J. S., Beran, M. J., & Washburn, D. A. (2006). Dissociating uncertainty responses and reinforcement signals in the comparative study of uncertainty monitoring. Journal of Experimental Psychology: General, 135, 282-297. doi: 10.1037/0096-3445.135.2.282 Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press. Zentall, T. R., & Stagner, J. P. (2010). Pigeons prefer conditional stimuli over their absence: A comment on Roberts et al. (2009). Journal of Experimental Psychology: Animal Behavior Processes, 36, 506-509.
Author note 12 Correspondence concerning this article should be addressed to Mike Le Pelley, School of Psychology, University of New South Wales, Sydney NSW 2052, Australia. Email: m.lepelley@unsw.edu.au