Is Cognitive Science Special? In what way is it special? Cognitive science is a delicate mixture of the obvious and the incredible

Transcription

1 Sept 3, 2013 Is Cognitive Science Special? In what way is it special? Zenon Pylyshyn, Rutgers Center for Cognitive Science Cognitive science is a delicate mixture of the obvious and the incredible What s in the mind that we may know it? (cf Shakespeare s What's in the brain, that ink may character? ) Granny was almost right: Behavior really is governed by what we believe and what we want (together with the mechanisms for representing and for drawing inferences from these) It s emic, not etic properties that matter Kenneth Pike What determines our behavior is not how the world is, but how we represent it as being As Chomsky pointed out in his review of Skinner, if we describe behavior in relation to the objective properties of the world, we would have to conclude that behavior is essentially stimulus-independent Every behavioral regularity (other than physical ones like falling) is a function of how we interpret and represent the world what we see things as A lesson learned from anthropology: It s emic states that matter! 1

2 The central role of representation presents some serious problems for a natural science What our beliefs are about is what matters But how can the fact that a belief is about tsome particular thing have an observable consequence? e.g. How can the presence of holy grail in a belief determine behavior when the holy grail does not exist? In a natural science if X causes Y then X must exist and be connected to Y by natural law(s)! It s even worse than that; even when X exists, it is not X s physical properties that are relevant to cognition! e.g., the North Star & navigation This dilemma is sometimes referred to as Brentano s problem or The Problem of Intentionality What determines what we do is what our mental states are about, but aboutness is not a category of natural science. It does not occur in any natural law. That is why Brentano (and Wolfgang Kӧhler after him) concluded that psychology was beyond the grasp of natural science. WHAT WE HAVE LEARNED: Intelligent systems behave the way they do because of what they represent (nb that they represent things as being) But in order to function under physical principles, the representations must have physical properties that encode the content of the representation How to encode knowledge in physical properties is by first encoding it in symbolic form (Symbolic Logic and Proof Theory tells us how) and then instantiating those symbolic codes physically (computer science tells us how) Cognitive Science and the Tri-Level Hypothesis Intelligent systems are inherently organized at three (or more) distinct levels 1. The physical or biological level 2. The symbolic or syntactic level 3. The knowledge or semantic level This means that explaining different regularities may require appeal to properties and structures at different levels 2

3 Does intentionality (and the trilevel hypothesis) only apply to high-level processes such as reasoning? An example from color vision: i Red light and yellow light mix to produce orange light This remains true for any way of creating red light and yellow light: e.g. yellow may be light of 580 nanometer wavelength, or it may be a mixture of light of 530 nm and 650 nm wavelengths. Does intentionality (and the trilevel hypothesis) only apply to high-level processes such as seeing-as? An example from vision linkages among interpretations provides the basis for the natural constraint methodology So long as one light looks yellow and the other light looks red the law of color mixing will hold. Skip to 17? If cognitive processes are at a different level of organization from the physical level, how can we ever discover what they are? We are limited only by the imagination of the experimenter, e.g., Relative complexity evidence (RT, error rates ) Intermediate state evidence Eye tracking Verbal Protocols Event Related Potentials (EEG) fmri, clinical observations of brain damage, Psychophysical methods (SDT), Etc Example: Sternberg s rapid memory search Give subjects a set of 2 to 6 letters to memorize (this is the memory set). ) Next tasks will refer to these letters. Present subjects with a single letter (this is the probe). Subjects must respond or to indicate whether the probe letter was the same as one of the memory set letters (usually respond by pressing a button, the latency of which is recorded). Use Reaction Time (RT) as a measure of how much processing went on during the search. Plot RT against Independent (manipulated) Variables (e.g. number of letters in the set, whether the probe was in the memory set or not present vs absent ) Results 3

4 Results of the Sternberg memory search task What do they tell us about how people do it? Is this Input-Output equivalent or is it strongly equivalent to human performance? To get at the details of the process, measure reaction times. Suppose we find one of thee two possible outcomes: Results of the Sternberg memory search task What do they tell us about how people do it? Is this Input-Output equivalent or is it strongly equivalent to human performance? To get at the details of the process, measure reaction times: Option 1 Option 2 ction time (seconds) Reac Results of the Sternberg memory search task What do they tell us about how people do it? Is this Input-Output equivalent or is it strongly equivalent to human performance? Exhaustive search Self-terminating search Two other considerations that are special to cognitively determined behavior 1. The Cognitive Penetrability of most cognitive processes. A regularity that is based on knowledge can be altered in a coherent way by providing new information, regardless of the physical form of that information (pictorial, linguistic, ascii ). 2. The critical role of Cognitive Capacity. Because of an organism's ecological or social niche, only a small fraction of its behavioral repertoire is ever actually observed. An adequate cognitive theory must account for the behavioral repertoire that is compatible with the organism's structure or its architecture. We call this total behavioral repertoire its cognitive capacity. 4

5 Aside on scientific explanation The difference between explanations that appeal to (relatively fixed) architecture and those that appeal to (relatively malleable) representations. Suppose we observe and summarize (e.g., in mathematical equations) some reliable behavioral regularity. What does it tell us about the nature of the system, or how it works? Note that a systematic record might allow us to extrapolate and make predictions. But the reason we study science (esp cognition) is not mainly to make predictions, but to understand how something works. An example to illustrate the concept of capacity Clarifying the Obligatory requirement The parable of the found mystery box A Cognitive Scientist, out walking in a field one day, comes upon a black box which happens to have a meter and recorder that records the meter s changes over time (like an EEG record). The Cognitive Scientist examines lots of records generated by the box and finds the pattern to be quite systematic. It follows the following regular pattern: An illustrative example: Mystery Code Box An illustrative example: continued Careful study reveals that pattern #2 only occurs in this special context when it is preceded by pattern A What does this behavior pattern tell us about the nature of the box? What does this behavior pattern tell us about the nature of the box? 5

6 Regularities in behavior may tell us nothing about how a system works because the pattern may be due to one of two different sources: 1. The inherent nature of the system (to its structure or its capacity) its architecture. 2. The nature of what the system represents (what it knows ) about the outside world. It this example, the reason why this pattern was observed, as opposed to some other pattern that the system might equally have exhibited, is external to the system itself. Note: Science is about understanding what could be it is about counterfactuals not what is typical. Where it matters: Application of the architecture vs knowledge distinction to understanding what goes on when we reason using mental images. Examples of behavior regularities attributable to tacit knowledge Mental color-mixing example Mental Color mixing? The effect of image size? Scanning mental images? This example is typical of almost all research on how mental images are used in reasoning. They rely on people s use of what they tacitly know would happen in the world and then making their imaginings mimic that. 6

7 The important distinction between the architecture and the content of representations It is important to ask why certain patterns (e.g. color mixing) occur whether the pattern is caused by fixed properties of the architecture t or because of properties of what is represented (i.e., what the observer tacitly knows about the behavior of that which is represented) If they occur only because the theorist says they do, then score that as a free empirical parameter (a wild card). The important consequence is that if we allow one theory to stipulate why that pattern occurs without there being a principle reason for it, then any other theory can stipulate the same thing. Such unconstrained theories explain nothing. This failure of image theories is quite general all picture theories suffer from the same lack of principled constraints. A famous example: Mental Scanning Hundreds of experiments have now been done demonstrating that it takes longer to scan attention between places that are further apart in the imagined scene. In fact the time-distance relation is linear. These have been reviewed and described in: Denis, M., & Kosslyn, S. M. (1999). Scanning visual mental images: A window on the mind. Cahiers de Psychologie Cognitive / Current Psychology of Cognition, 18(4), Rarely cited are experiments by Liam Bannon and me (described in Pylyshyn, 1981) which I will summarize for you. The central problem with imagistic explanations What is assumed in the mental picture explanation of mental scanning? In actual vision, it takes longer to scan a greater distance in the world because real distance, real motion, and real time is involved, therefore this equation holds due to natural law: Time = distance speed But what ensures that a corresponding relation holds in an image? The natural answer is: Because the image is laid out in real space! But what if that option is blocked for empirical reasons? You might try to weasel out byappealing to a Functional Space, which theorists liken to a matrix data structure in which some pairs of cells are closer and others further away, and to move from one to another it is natural that you pass through intermediate cells. But such constraints are not inherent in the architecture if they can be changed by changes in beliefs which they invariably can be! Studies of mental scanning Does it show that images have metrical space? Latency (secs) scan image imagine lights show direction Relative distance on image (Pylyshyn & Bannon. Described in Pylyshyn, 1981) Conclusion: The image scanning effect is Cognitively Penetrable i.e., it depends on Tacit Knowledge. 7

8 THE INTENTIONAL FALLACY failure to recognize an ambiguity: A representation of X with property P can mean: [1] A representation of (X with property P) In this case it is X that has property P, A representation of a red circle may be neither red nor round! [2] (A representation of X) with property P In that case the representation has property P A part of a picture of a red circle is usually red and circular So why does it not feel like we are doing computations? Because the content of our conscious experience is a very poor guide to what is actually going on that causes our behavior. Science is concerned with causes, not just correlations. Because we can t assume that the way things seem has much to do with how it works (e.g., language understanding) As in most sciences, the essential causes are far from obvious (e.g., How do we know that the earth goes around the sun rather than the other way round?). What does it look like? In the case of cognition, what is going on is a delicate mixture of the obvious (what Granny or Shakespeare knew about why people do what they do) and the incredible 8

9 Why do so many people deny these obvious facts about mental imagery? The power of subjective experience (phenomenology). Subjective experience does not cause behavior; only physical properties (basic properties of science) have causes! The failure to make some essential distinctions o Content vs form (the property of images vs the property of what images are about) {compare the code box example} o An image of X with property P can mean (An image of X) with property P or An image of (X with property P) Capacity vs typical behavior: Architecture vs knowledge Are all the things we thought were due to internal pictures actually due to tacit knowledge? Other reasons for imagery phenomena: Task demands: Imagine that X = What would it be like if you saw X happening? Are there pictures in the brain? No evidence for pictorial it ildisplays sufficient ffii to explain visual or imaginal phenomena So what is in the brain? Shakespeare asked What's in the brain, that ink may character? The best hypothesis so far (i.e., the only one that has not been shown to be clearly wrong) is that the brain is a species of computer in which representations of the world are encoded in the form of symbol structures, and actions are determined by calculations (i.e., inferences) based on these symbolic encodings. 9