Synesthesia and Consciousness

Transcription

1 Synesthesia and Consciousness Whisper by Sohrab Sepehri Hadjar Homaei May 2007 Introduction Synesthesia (also spelled synæsthesia or synesthesia, plural synesthesae or synaesthesae) from the Greek roots syn, meaning together, and aisthesis, or perception, is a neurological condition in which otherwise normal people experience the blending of two or more senses. [1] There are many different theories on this phenomenon such as cross activation between two normally separate areas of the brain when they educe activity in each other. Explorations in the mechanisms involved in synesthesia, not only helps to uncover mystic ideas behind this phenomenon, scientists are also learning about how the brain in general processes sensory information and uses it to make abstract connections between seemingly unrelated inputs. [1] In this paper, we discuss some of possible neuropsychological theories on synesthesia and propose an experiment to investigate the relation between synesthesia and mechanisms of conscious experience.

2 Neural Models of Synesthesia Cytowic in The man who tasted shapes concludes that synesthesia is not a higher cortical function in the usual sense. His argument is basically derived from a phenomenological analysis and reasoning by analogy to more common phenomena that were qualitatively similar to the experience of synesthesia. Disregarding what the nature of the link between a stimulating sensation and the synesthetically-perceived one might be, proposed that the level of this unknown link occupied a low to intermediate level of the neuraxis, rather than a higher level involving more mental mediation. So far, there have been two, rather parallel debates on the neural basis of synesthesia. The first of these debates takes place in the neuro-physiological level and concentrates on the question of whether synesthetic experience is caused by a failure of neural pruning or some kind of dis-inhibition (Ramachandran and Hubbard [2001a], Grossenbacher and Lovelace [2001]).The second debate takes place in the architectural level. Three architectural models have been suggested, which Ramachandran and Hubbard [2003] refers to them as local cross-activation, re-entrant processing, and long-range disinhibited feedback. Although these two views are logically independent, not all possible combinations of these mechanisms have been yet proposed. Local Cross-activation Based on the fact that the visual word form area in the brain is close to hv4 region which processes color, it has been proposed that grapheme-color synesthesia may be caused by direct cross-activation between these two adjacent brain areas (Hubbard et al. [2005a], Ramachandran and Hubbard [2001a] [2001b]). Ramachandran and Hubbard [2003] suggest that synesthesia occurs based on a mechanism of cross-activation similar to that observed in phantom limb patients which leas to systematic perceptual experiences, that are reproducible and involuntary. This hypothesis is based on previous studies proposing that phantom limb sensations may arise through cortical reorganization in one who has had a limb or body part surgically removed. These cortical-to-cortical connections led to systematic reproducible and involuntary perceptual experiences of feeling the removed limb stimulated through stimulation of the facial nerves belonging to the missing limb that are still present. One possible mechanism for this could be the extensive connections between inferior temporal area and V4 before birth. For fetal macaques, approximately 70% to 90% of the connections to V4 are from higher areas, while in the adult this range decreases to 20% to 30%. A failure in pruning of these prenatal pathways, could lead to connections between the VWFA and hv4 to persist into the adulthood life and cause the experience of color when viewing graphemes. Long-Range Dis-inhibited Feedback Other studies have suggested that synesthesia might be the result of dis-inhibited feedback from a multi-sensory nexus such as the temporo-parietal-occipital junction

3 (Grossenbacher and Lovelace [2001]). An interesting evidence supporting this model comes from the study of a patient, PH, who became blind at the age of 40 as a result of Retinitis Pigmentosa (Armel and Ramachandran [1999]). After 2 years of blindness, PH began reporting that tactile stimuli educed the impression of seeing visual movement. Also, the intensity of the stimulation necessary to induce synesthetic sense was greater when for example his hand was in front of his face than when it was behind his face that suggests some type of top-down multi-sensory activation, maybe induced by parietal structures. Another evidence supporting the dis-inhibited feedback theory is that at least some people report synesthetic experiences while under the influence of psychedelics as described in the previous paper. However, the experiences of real synesthetes, despite of the apparent similarities with drug-induced synesthesia, may be caused by different other mechanisms (Hubbard and Ramachandran [2003]). The effects of psychedelics tend to be systemic, while real synesthetes usually have exact mappings like the letter X always being some particular shade of color Y. In the absence of some mechanism to account for these phenomenological differences, it is difficult to see how generic and drug-induced synesthesia could be cause by a common mechanism. Re-Entrant Processing Something of a hybrid model has been suggested (Myles et al. [2003], Smilek et al., [2001]), in which grapheme-color synesthesia has been suggested to be caused by abnormal re-entrant processing which is consistent with the dis-inhibited feedback model. Smilek et al. propose that in addition to the forward pass of neural activity from V1 to V4, to posterior and then anterior inferior temporal regions (PIT and AIT, respectively), abnormal feedback from AIT to PIT and V4, leads to the experience of synesthetic colors. The major supporting evidence in favor of this theory over the cross-activation theory is that visual context and meaning influence the experienced colors in synesthesia (Dixon and Smilek [2005], Myles et al. [2003]). Hubbard and Ramachandran [2005] argues that the presence of behavioral top-down modulations cannot distinguish between the local cross-activation and re-entrant models of synesthesia because neither model has been specified with sufficient precision. Feedback is everywhere in the visual system but there are different mechanisms by which it can affect the emergence of synesthetic colors. The main difference between the models is how this feedback is supposed to produce synesthetic colors. To explain the presence of contextual modulations, the Re-entrant Processing model suggests that neural signals propagate back to V4 (labeled 1 in Figure 1A, from Hubbard and Ramachandran [2005] ), and in parallel, this activation affects the recognition of the original grapheme in PIT in that way leading to the effect of meaning on the synesthetic color.

4 Figure 1. The Cross-activation model can explain these contextual modulations by assuming that the same feedback mechanisms that can cause top-down modulations in non-synesthetes are also present in synesthetes. Once those mechanisms are in place, activation from AIT can directly bias the firing of neurons in PIT, such that the responses of these neurons differ. The different pattern of PIT (VWFA) neuronal firing will cross-activate a different population of V4 neurons, leading to both the percept of a different grapheme and a different synesthetic experience (labeled 2 in Figure 1B). Multiple Neural Mechanisms Obviously, these theories are not mutually exclusive. In studies of phantom limbs, there is a similar debate between unmasking and sprouting that can lead to explanations for this phenomenon. In some cases, phantoms sensations are experienced less than 24 hr after amputation, which implies the unmasking of connections that had been inhibited. In spite of this, longer term studies of amputees show that the organization of the point-topoint correspondence between facial trigger zones and phantom limb sensations in the arm develops slowly after amputation, which suggests that new neural connections are being developed. A similar relationship between preserved over-connectivity and dis-inhibited feedback may be there in synesthesia, in charge of the observed mixture of occasional synesthetic effects in the general population such as synesthetic language metaphors and the specific experiences of congenital synesthetes.

5 Different Neural Mechanisms for Different Synesthetes Another possibility is that a one size fits all approach may fail to capture the variability in synesthetic experiences. Different neural theories have focused on different types of synesthesia, with the local cross-activation and re-entrant feedback theories focusing on grapheme-color synesthesia, whereas the feedback models have focused on word-color and tone-color synesthesia. It is quite likely, given that graphemes, phonemes, music, and colors are processed by different brain regions, that forms of synesthesia have different architectural substrates. However, the fact that synesthetes within the same family may inherit different forms of synesthesia (Ward and Simner, 2005) suggests that common neuro-physiological mechanisms may be shared across different forms of synesthesia. Consciousness According to the Stanford encyclopedia of Philosophy, the words conscious and consciousness terms include a broad variety of mental phenomena. They are used with a diversity of meanings, and the adjective conscious can be applied both to whole organisms to account for creature consciousness and to particular mental states and processes for state consciousness. Any cognitive system can be regarded as conscious in a number of different senses. Sentience it is conscious in the generic sense of being a sentient creature, capable of sensing and responding to its world. Wakefulness In this type of consciousness requires the organism to actually be use such a capacity rather than just having the ability or tendency to do so. Therefore one might consider it conscious only if it is awake and alert and not conscious when asleep or in any level of coma. Self-consciousness This sense of conscious which might be the most demanding one defines conscious organisms as those that are not only aware but also aware that they are aware (Carruthers [2000]). What it is like - According to Thomas Nagel [1974] a being is conscious only if there is something that it s like to be that being. This is commonly referred to as what it is like criterion that captures a more subjective notion of being a conscious organism emphasizing on the creature's mental or experiential point of view. Subject of conscious states - Another alternative view is defining the notion of a conscious organism in terms of conscious states. That is, one might first define the requirements for a conscious mental state to be considered conscious, followed by defining being a conscious organism in terms of having these conscious states. Therefore the concept of a conscious organism will depend on the particular account one gives of conscious state.

6 Transitive Consciousness This notion of consciousness according to Rosenthal [1986] emphasizes stimuli of different senses of which the organism can be conscious of. The distinction between transitive and intransitive consciousness is that transitive notion usually involves some object at which consciousness is directed. State consciousness - The notion of a conscious mental state also has a variety of distinct but inter-related meanings. There are at least six major options. States one is aware of - A conscious mental state is simply a mental state one is aware of being in (Rosenthal [1986], [1996]). Conscious states in this sense are are mental states that are themselves about mental states so they have a meta-mentality or metaintentionality form in fact. Unconscious desires and thoughts can be caused by simple lack of attention or deeper psychoanalytical causes. Qualitative states Conscious states can also be regarded as conscious in a quite different and more qualitative sense. One might regard a state as conscious only if it has or involves qualitative or experiential properties of the sort often referred to as qualia or raw sensory feels or in simpler words the ways things seem to us. They can be defined as qualities or feelings, like redness or pain, as considered independently of their effects on behavior and from whatever physical circumstances give rise to them. Phenomenal states This notion of mental state includes the qualitative properties of the conscious experience including our existence as the center of this experience. In addition to the sensory qualia the phenomenal consciousness also includes much of the spatial, temporal and conceptual organization of our experience of the world and of us as agents in it. Interestingly the hard problem of consciousness was discussed by David Chalmers in 1996, dealing with the issue of "how to explain a state of phenomenal consciousness in terms of its neurological basis" (Block [2004]). What-it-is-like states - Consciousness in any sense can have a what is it like state linked to it simply counting a mental state as conscious in this sense only if there is something that it is like to be in that state. It only adds a first-person or internal conception to any notion of conscious experience. Access consciousness In this notion of consciousness a mental state is conscious if other senses have access to its content. This is a rather functional approach to consciousness because for example having visual conscious state and feeling what it is like to see visual stimulus does not matter much unless it s accessible for verbal report, reasoning, and the control of behavior. Problem: Qualia in Synesthetes After this short review of consciousness, maybe it s the right place to discuss the general problem that we are addressing in this paper. The abnormality in synesthete perception occurs when a stimulus of the inducer sense is perceived. As a result the perception of the

7 inducer stimulus is intact but there is an additional experience on the concurrent sense. All brain imaging, recording and other physiological studies basically involve the perception of concurrent sense in the presence of an inducer stimulus, except for bidirectional synesthetes. Although the target debate in the study of synesthesia is to find the neural mechanism of synesthetic perception, but investigating the differences in qualities of the conscious experience in concurrent sense synesthetes and normal people might provide us with some clues to better understand the mechanism underlying synesthetic perception. Assuming it might be possible for all people with normal sight, experience conscious visual states with similar qualities (which might be a false assumption in some aspects) or share some similar qualities, it is possible that what it s like to be a tone-color synesthete, be a different experience. Some argue that under no circumstances can empirical science speak of consciousness as such, while others claim that the scientific goal is "knowing what it is like to be a bat" to share an organism's conscious experience (Nagel, 1974). The purpose of our experiment is going to be investigating the differences between the qualities of these two conscious states that are accessible. As we will briefly discuss later, most tests of access consciousness involve comparing the current experience with the memory of the same person s similar subjective past experiences associated with some arbitrary concept that is used to report the experience. For example the subject reports that the red circle on the screen in red because it produces a similar experience to perceiving the color of blood (referring to the common definition of redness in dictionaries as color similar to that of blood ) so because of their subjective nature, many qualities of the color perception are not truly accessible. Crick and Koch [1995] propose that the function of visual consciousness is to produce the best current interpretation of the visual scene and to make it available for a sufficient time to the parts of the brain that contemplate, plan, and execute voluntary motor outputs (including language). The reason that we chose visual consciousness, as the concurrent sense to focus on is because it is experimentally the most accessible and the best understood. The most popular belief about the neural basis of consciousness is that consciousness is an emergent property of a very large pool of neurons in a network. An alternative theory is that there is a set of special types of neurons in the cortex and associated systems, such as the Thalamus and the Basal Ganglia, that represent the ultimate neuronal correlate of consciousness (NCC), and the activity of an certain subsets of them is both necessary and sufficient to give rise to one particular conscious experience or percept associated with that subset (Crick and Koch [1995]). One of the basic schemes of finding NCC is using a bi-stable percept that alters the state of consciousness without the visual stimulus being changed. This phenomenon is called binocular rivalry. The neuron whose firing pattern from single neuron recording matches the pattern of switches in conscious states is a part of neurons in NCC. Unfortunately, this trend in study of consciousness mostly requires single cell recordings that are not possible to be done on human.

8 Although we talked about conscious state alters between two modes in binocular rivalry, it is not simply an on-off switch. Conscious experiences have a complex structure and include complex representational contents. A phenomenal conscious state usually includes various perceptual experiences, physical bodily and emotional sensations. Each of these aspects of the conscious state can have a complex structure themselves. For example, a typical visual experience has an internal structure representing objects with many different colors, shapes and patterns in different degrees of detail. Figure 2. (a) Binocular rivalry can be demonstrated by placing a pen between yourself and the screen. Keep you eye on the tip of the pen and notice the two bars merge. You may need to slowly move the pen from the screen towards you. (b) Result of (a). The content of consciousness is not a new concept. They have been studied throughout the history of psychology, for example psychophysics studies that focus on particular aspects of these contents, such as the subjective brightness perception in a visual experience, and correlate it with properties of the associated stimulus [Weber, Fechner,?]. This provided a basic link between first-person data about sensory experience and third-person data about a stimulus that we are going to discuss late. Later works in psychophysics and Gestalt psychology have taken an approach of the same general type, examining particular features of perceptual experience and analyzing how these co-vary with properties of their associated stimulus. More recent research also tries to take even more contents of consciousness in account, For example, research on visual illusions often uses subjects' first person reports to characterize the structure of perceptual experiences. Research on attention (Mack and Rock [1998], Treisman [2003]) tries to describe the structure of perceptual experience content inside and outside the focus of attention. Other researchers look into the contents of consciousness in the other domains such as mental imagery (Baars [1996]), emotional experience (Kaszniak [1998]), and the stream of conscious thought (Pope [1978], Hurlburt [1990]).

9 Chalmers [2004] argues that the distinctive task of a science of consciousness is to systematically integrate two key classes of data into a scientific framework: third-person data about behavior and brain processes, and first-person data about subjective experience. Observing the conscious subject from the third-person point of view provides us a range of specific behavioral and neural phenomena. Opposed to the first-person point of view that gives a range of specific subjective phenomena. Both types of phenomena can be used as data for characterizing a conscious state. Third-person data is basically based on the behavior and the brain processes of conscious systems. This is the typical type of data for most cognitive psychology and of cognitive neuroscience experiments. Here s a list of third-person types of data that can be useful for science of consciousness: (Chalmers [2004]) Perceptual discrimination of external stimuli The integration of information across sensory modalities Automatic and voluntary actions Levels of access to internally represented information Verbal reportability of internal states The differences between sleep and wakefulness First-person data is based on the subjective experiences of conscious systems. We as conscious systems all know for a fact that these types of experiences exist: we can use our own subjective experience or gather information provided by monitoring subjective verbal reports of other conscious subject s experience. These phenomenological data provide the distinctive subject for the science of consciousness. Here s a list of firstperson types of data that can be useful for science of consciousness: (Chalmers [2004]) Visual experience (e.g. the experience of color and depth) Other perceptual experiences (e.g. auditory and tactile experience) Bodily experiences (e.g. pain and hunger) Mental imagery (e.g. recalled visual images) Emotional experience (e.g. happiness and anger) Occurrent thought (e.g. the experience of reflecting and deciding) Both third-person data and first-person data need explanation preferably within a unified and integrated framework. For example in our work, if we observe a subject attending to some image, relevant third-person data include those concerning the nature of the visual stimulus such as size, hue, saturation, duration of exposure, etc, how it affects on the eye and the visual cortex of the subject. (Which is not practical for us to investigate), various behavioral responses by the subject, and any verbal reports the subject might produce. But that is not all that needs explanation. We will also need first-person data to be able a complete explanation of the phenomena.

10 In order to investigate different aspects of the color vision in synesthetes with visual type of experience as their concurrent sense, we came up this experiment. Experiment. Temporal resolution There are minimum requirements for different properties of a visual stimulus to be perceivable by human subjects, such as minimum exposure time, hue, saturation and intensity. If the stimulus does not meet those thresholds, subject might not be able to confidently report the experience although if forced to respond we might observe a subliminary effect in the responses. So in this experiment both synesthete and normal subjects are exposed to a vide range of visual stimuli in random manner. In each run, a new saturation and intensity indexes are chosen and for each stimulus in the run a hue value is picked. Exposure times are increased until the level subjects report confidence in their response. Figure 3. Color panel used by subject to pick the color that matches the color of stimulus The final result analysis includes comparing the difference between the accuracy of responses and duration threshold for synesthete and normal subjects and how also the difference between changes in these thresholds relative saturation and intensity in stimuli. It s expected that synesthete subjects have lower thresholds for color perception. References: [1] Edward M. Hubbard, V.S. Ramachandran, Neuromechanisms of synesthesia, Neuron, Vol. 48, , November 3, 2005 [2] V.S. Ramachandran, Edward M. Hubbard, Hearing Colors, Tasting Shapes, Scientific American, p 53-59, 2003 [3] V.S. Ramachandran, A brief tour of human consciousness, Pi Press, ISBN: , 2003 [4] James A. Schirillo, A Synesthesia Experiment: Consciousness of Neural Activity, The Man Who Tasted Shapes, The MIT Press, ISBN: , 1998 [online copy]

11 [5] Richard Cytowic, The Man Who Tasted Shapes, New York: Tarcher/Putman, 2003 [an online review on the book by the author] [6] Bernard Baars, A thoroughly empirical approach to consciousness, [6] Chalmers, D. How can we construct a science of consciousness?, M. Gazzaniga (ed.), The Cognitive Neurosciences III MIT Press, 2004 [CogNet copy] [7] Chalmers, D. Facing up to the problem of consciousness. Journal of Consciousness Studies, Vol 2, P [8] Chalmers, D. The Conscious Mind. Oxford: Oxford University Press [online copy] [9] Crick, F. Koch, C., Toward a neurobiological theory of consciousness. Seminars in Neuroscience, Vol 2, P , 1990 [online copy] [10] Koch, C., Neurobiology of Consciousness, Neural Basis of consciousness lecture course notes from [11] John Taylor, Modeling consciousness [12] Rick Grush, Perception, imagery, and the sensorimotor loop [13] Tye, M., Consciousness, Color, and Content., MIT Press., 2000 [CogNet copy] [14] Michael Polanyi, The structure of consciousness [15] Robert Van Gulick, Consciousness, Stanford encyclopedia of philosophy, 2004 [16] Metzinger, T. ed., Neural Correlates of Consciousness: Empirical and Conceptual Questions. MIT Press [CogNet copy]