UNIVERSITY OF CALIFORNIA, SAN DIEGO The Role of the Neostriatum in the Execution of Action Sequences A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Cognitive Science by John Randall Gobbel Committee in charge: Professor Martin I. Sereno, Chair Professor Jeffrey Elman Professor David Zipser Professor Philip Groves Professor Terrence J. Sejnowski 1997
Copyright John Randall Gobbel, 1997 All rights reserved
The dissertation of John Randall Gobbel is approved, and it is acceptable in quality and form for publication on microfilm: University of California, San Diego Chair 1997 iii
Table of Contents Signature Page:... iii Table of Contents... iv List of Figures and Tables... vii Acknowledgments... ix Vita...x Abstract of the Dissertation... xi Chapter 1 Introduction...1 1.1 Overview...1 1.2 Overview of the basal ganglia and motivation for modeling...2 Chapter 2 Related work...5 2.1 Abstract connectionist models...5 2.1.1 Berns & Sejnowski...5 2.1.2 Borrett et al....6 2.1.3 Brotchie et al...6 2.1.4 Cohen...6 2.1.5 Dominey & Arbib...7 2.1.6 Jamieson...8 2.1.7 Mitchell et al....8 2.2 Biophysically-based models...9 2.2.1 Connolly & Burns...9 2.2.2 Porenta...9 2.2.3 Wickens...10 2.2.4 Wilson...10 2.2.5 Woodward et al....11 2.3 Summary...12 Chapter 3 Background...14 3.1 Nuclei of the basal ganglia...14 3.2 Anatomy of the basal ganglia...15 3.2.1 The neostriatum...18 3.2.2 The internal segment of the globus pallidus (GPi) and pars reticulata of the substantia nigra (SNr)...25 iv
3.2.3 The external segment of the globus pallidus (GPe)...26 3.2.4 The pars compacta of the substantia nigra (SNc)...26 3.2.5 The subthalamic nucleus (STN)...26 3.3 Basal ganglia disorders...27 3.3.1 Parkinsonian syndromes...27 3.3.2 Huntington s disease...32 3.4 Behavioral studies in rats and primates...33 3.4.1 Hikosaka: characteristics of striatal cells...33 3.4.2 Schultz: characteristics of dopaminergic cells...35 3.4.3 Packard, Hirsh, & White: the striatum and procedural learning...35 3.5 Physiological details of the neostriatum...36 3.5.1 The medium spiny neuron...36 3.5.2 Ion channels in the medium spiny cell...38 3.5.3 The cholinergic interneuron...40 3.5.4 Ion channels in the cholinergic cell...41 3.5.5 Modulation...41 Chapter 4 Compartmental models of medium spiny and cholinergic neurons...44 4.1 Overview...44 4.1.1 Voltage-gated currents in the model spiny cell...46 4.1.2 The model cholinergic cell...47 4.2 Modulation in the model...48 4.2.1 D 1, D 2, and dopamine...49 4.2.2 The switching process...50 4.3 Experiments with the medium spiny neuron model...51 4.3.1 Basic results...52 4.3.2 Effects of I KFIR activation slope changes...54 4.3.3 Cholinergic effects in the model medium spiny neuron...54 4.4 A possible functional role for two-state behavior in medium spiny neurons...55 Chapter 5 Network models...57 5.1 Overview...57 5.1.1 Basic steps in set-shifting and sequence execution...58 5.1.2 One-trigger and two-trigger architectures...59 v
5.2 The single-level network...61 5.2.1 Multilevel networks...63 5.2.2 Variations of multilevel networks...65 5.3 Sequence execution with multilevel network models...66 5.4 Simulation of Parkinson s disease...67 5.4.1 Method for lowering dopaminergic activation...67 5.4.2 Main effects of reduced dopamine...68 5.4.3 Different failure modes in one- and two-trigger architectures...68 5.4.4 The multilevel network models predict striatal and cortical hyperactivity...70 5.4.5 Hierarchy and the etiology of idiopathic Parkinson s disease...70 5.4.6 Postencephalitic Parkinson s disease and motor genius...71 5.4.7 Huntington s disease...72 5.5 Perspectives on network simulation results...73 5.6 Final thoughts...74 Appendices Appendix A. A Biophysical Model of the Neostriatum. I: Dopaminergic Modulation of Dynamically Bistable Membrane Potential in Neostriatal Medium Spiny Cells...76 Appendix B. A Biophysical Model of the Neostriatum. II: Control of Action Sequences and Parkinson s Disease...114 Bibliography...174 vi
List of Figures and Tables FIGURE 1. Large structures of the basal ganglia in relation to the cortex... 15 FIGURE 2. Block diagram of the basal ganglia and related structures... 17 FIGURE 3. Summary of basal ganglia connections... 18 FIGURE 4. Loops through the basal ganglia,... 19 FIGURE 5. Topographic organization of striatal input... 20 FIGURE 6. Cytoarchitecture of the neostriatum... 21 FIGURE 7. Memory-based saccade paradigm... 34 FIGURE 8. Membrane potential fluctuations in a striatal medium spiny cell... 37 TABLE 1. Time constants for synaptic conductances... 45 TABLE 2. Channels vs. cell types... 47 FIGURE 9. A summary of modulatory effects... 49 FIGURE 10. Membrane potential in spiny cells of different context/input combinations... 52 FIGURE 11. Architecture of the single-level network.... 61 FIGURE 12. Two variations on the basic multilevel architecture... 64 FIGURE 13. A summary of modulatory effects on voltage-gated currents in the model medium spiny neuron... 106 FIGURE 14. Membrane potential in model spiny cells of different context/input combinations... 107 FIGURE 15. Membrane potential fluctuations in a striatal medium spiny cell.... 108 FIGURE 16. Channel current as a proportion of peak conductance... 109 FIGURE 17. The effect of changing the slope of the activation curve of I KFIR... 110 FIGURE 18. The effect of cholinergic modulation on the model medium spiny cell... 111 TABLE 3. Synaptic conductance parameters... 112 TABLE 4. Voltage-gated channel Hodgkin-Huxley equations... 112 TABLE 5. Medium spiny cell parameters... 113 TABLE 6. Cholinergic cell model parameters... 113 FIGURE 19. A minimal network for shifting a model medium spiny neuron between Up and Down states.... 160 FIGURE 20. The asymmetry found by Ragsdale and Graybiel... 161 FIGURE 21. A possible pathway for hierarchical control of action in the corticostriatal system... 162 FIGURE 22. The conceptual architecture of a recurrent network model for executing sequences of actions.... 163 FIGURE 23. Architecture of the single-level network... 164 FIGURE 24. Minimal network for demonstrating contextualized responses... 165 vii
FIGURE 25. Two variations on the basic multilevel architecture... 166 FIGURE 26. Connectivity of the two-trigger multilevel network model... 167 FIGURE 27. Connectivity of the one-trigger multilevel network model.... 168 FIGURE 28. Membrane potential traces during running of the single-level network... 169 FIGURE 29. Execution trace for the two-trigger multilevel network.... 170 FIGURE 30. Execution trace for the one-trigger multilevel network.... 171 FIGURE 31. A trace of execution of the two-trigger multilevel network with lowered levels of dopaminergic activation... 172 FIGURE 32. A trace of execution of the one-trigger multilevel network with lowered levels of dopaminergic activation... 173 viii
Acknowledgments I first want to thank my adviser, Marty Sereno, for his excellent guidance, helpful suggestions, and thoughtful feedback through the years I have worked with him. It has truly been a pleasure. David Zipser is responsible for first pointing me in the direction of the basal ganglia, which turned out to be far more interesting than I would have suspected. Philip Groves served as my local expert on the basal ganglia, and patiently answered many questions that must have seemed rather naive. Jeff Elman and Terry Sejnowski have also provided helpful suggestions and perspectives. My thanks to the entire committee having reached this point, I can look back and say that I have actually enjoyed this process, and it has enriched my life immeasurably. Thanks to Charles J. Wilson for having been an extremely valuable source of information and perspectives on the current state of research on the basal ganglia, to the McDonnell-Pew Foundation for Cognitive Neuroscience for a graduate training fellowship which allowed me to focus on my research, and to the staff of the UCSD Department of Cognitive Science for their excellent support through the years. Finally, I would like to dedicate this dissertation to my best friend, soulmate, and wife, Ann Thymé-Gobbel, whose unfailing support during the last five years has been a true blessing. Through all-night printing sessions, last-minute bugfixes, and my somewhat less-than-mellow demeanor as deadlines approached, her unwavering love and devotion have been a source of strength when I needed it most. ix
Vita October 17, 1952 Born, Durham, North Carolina 1974 B.A., Communications/Political Science, Antioch College, Yellow Springs, Ohio 1974-1976 Systems Programmer, Resource One, San Francisco 1976-1979 Systems Programmer, Tymshare Inc., Cupertino, CA 1979-1990 Member of Programming & Research Staffs, Xerox PARC, Palo Alto, CA 1993 M.S., Cognitive Science, University of California, San Diego 1990-1996 Teaching & Research Assistant, Department of Cognitive Science, University of California, San Diego 1997 Ph.D., Cognitive Science, University of California, San Diego Publications Gobbel, J. R. (1996). Dopaminergic and cholinergic modulation in a biophysical model of the neostriatum. In J. M. Bower (Ed.), Advances in Computational Neuroscience: Proceedings of the Fourth Annual Computational Neuroscience Conference, Monterey, CA, July 1995. New York: Academic Press. Gobbel, J. R. (1995). A biophysically-based model of the neostriatum as a reconfigurable network. In M. Bodén & L.-E. Niklasson (Eds.), Current Trends in Connectionism: Proceedings of the Second Swedish Conference on Connectionism, Skövde, Sweden, March 1995. Hillsdale, NJ: Erlbaum. x
Abstract of the Dissertation The Role of the Neostriatum in the Execution of Action Sequences by J. Randall Gobbel Doctor of Philosophy in Cognitive Science University of California, San Diego, 1997 Professor Martin I. Sereno, Chair This dissertation explores the role of modulatory neurotransmitters and intrinsic cell membrane properties in the functioning of the neostriatum, the input structure of the basal ganglia, with particular attention to the execution of action sequences in normal functioning and in Parkinson s disease. A model of the principal projection cell type of the neostriatum, the medium spiny neuron, is constructed using compartmental modeling techniques. Voltage-gated channels and dopaminergic and cholinergic modulatory effects are modeled, using data obtained from published voltage-clamp studies of striatal medium spiny neurons to obtain parameter setting for Hodgkin-Huxley representations of individual ion channels. Simulation scenarios run with the model cells demonstrate that they duplicate patterns of bistable membrane potential fluctuations seen in biological medium spiny cells, including details of the shape of transitions between stable states. Membrane potential fluctuations in the xi
model cells are largely governed by the effects of the modulatory neurotransmitter dopamine. A further stage of modeling places the model medium spiny neuron in the context of a network simulation, whose architecture is patterned on the connectivity of the neostriatum and neocortex. Three stages of network architecture are explored, including a single-level network capable of stepping through sequences of actions in response to sensory inputs, and two hierarchically-organized two-level networks. The two-level networks are shown to be capable of executing combinations of sequences which cause difficulty for patients with Parkinson s disease. When simulated levels of dopamine are reduced, the two-level networks make errors in sequence execution which duplicate the results of behavioral experiments with Parkinson s disease patients. xii
Chapter 1 Introduction 1.1 Overview The basal ganglia consist of six interconnected nuclei located in the forebrain, diencephalon, and midbrain. They are known to play a crucial role in the learning and control of sequential motor behavior, and are now suspected to play a role in cognitive functions such as planning. This dissertation describes a new model of the neostriatum, the input structure of the basal ganglia. The model has two principal parts, a model of a single neostriatal projection neuron (Chapter 4, page 44), and a network model composed of these model neurons (Chapter 5, page 57). The model supports a novel theory of how the neostriatum encodes motor and perhaps cognitive set, which leads naturally to a more general theory of action control. Following this introduction and a discussion of related work (Chapter 2, page 5), the dissertation includes a discussion of anatomical background (Chapter 3, page 14). Basal ganglia nuclei other than the neostriatum are discussed in enough detail to make the explanation of striatal structure and function comprehensible. Following this, I present some evidence of the role of context in Parkinson's and Huntington's diseases, two well-known disorders involving the striatum, along with related evidence from animal studies ( 3.3, page 27). A discussion of the detailed physiology of the neostriatum leads into a description of the model. 1
2 1.2 Overview of the basal ganglia and motivation for modeling The basal ganglia are a set of nuclei situated between the neocortex and the thalamus. One substructure of the basal ganglia, the striatum, is the largest forebrain structure after the cortex itself. The basal ganglia play a central role in the control of voluntary movements, and recently have been implicated in cognitive functions such as planning and sequential action in general (Brown & Marsden, 1990). Disorders of the basal ganglia almost always produce some form of movement disorder, and one of the most common disabling diseases of old age, Parkinson s disease, is the result of a loss of dopaminergic innervation in the neostriatum, the input structure of the basal ganglia (Carlsson, 1959). In addition to their obvious movement difficulties, patients with basal ganglia disorders exhibit a variety of cognitive problems. The role of the basal ganglia in skill learning is also well-established (Packard et al., 1989, 1990; Packard & White, 1990, 1991). A theory of basal ganglia function is therefore a critical link in the search for an understanding of how the brain manages sequential action. Although there have been a number of proposals for the functional organization of the basal ganglia, few of these have been detailed enough to guide the construction of a simulation model, and of those that have been incorporated into working simulations, none has incorporated detailed models of the precise targets and dynamics of neuromodulators (Albin et al., 1989; Alexander & Wickens, 1993; Borrett et al., 1993; Brotchie et al., 1991; Brown & Marsden, 1990; Bunney et al., 1991; Cohen & Servan-Schreiber, 1992, 1993; Cohen et al., 1992; Dominey & Arbib, 1992; Grace, 1993; Graybiel et al., 1994; Groves, 1983; Houk et al., 1995; Jamieson, 1991; Mitchell et al., 1991; Morelli et al., 1991; Porenta, 1986; Swerdlow, 1995; Swerdlow & Koob, 1987; Wickens, 1993; Wickens & Arbuth-
3 nott, 1993). The model presented here uses recent physiological results to implement a model of the neostriatal medium spiny neuron and the overall architecture of automatized processing of sequences, including a detailed account of both dopaminergic and cholinergic modulatory effects. The basal ganglia as a whole are too complex to be simulated in a detailed computational model at this time, given both our current state of knowledge and the performance of current simulators. The largest substructure of the basal ganglia, the neostriatum, is interesting in its own right, however, and has several characteristics which make it a good starting place for modeling: Relative to the neocortex, the architecture of the neostriatum appears fairly straightforward. Despite their lack of direct connection to sensors and effectors, the principal neurons of the neostriatum have a reasonably clear relationship to phenomena in the world, firing in response to stimuli which indicate transition points in tasks (Hikosaka et al., 1989a). Projection cells make up the great majority of the neostriatal cell population. These cells tend to be extremely quiet, and apparently have little or no direct influence on each other (Jaeger et al., 1994). This simplifies network dynamics somewhat. Because the great majority of input to the basal ganglia passes through the neostriatum, understanding the neostriatum is a prerequisite for a more complete theory of the basal ganglia as a whole. The neostriatum is known to be a site of both long-term synaptic potentiation (LTP) and depression (LTD), and thus a possible site of Hebbian learning (Calabresi et al., 1992a, 1992b, 1992c; 1993; Lovinger et al., 1993).
4 Two well-studied disorders, Parkinson s disease and Huntington s disease, are accompanied by clearly observable changes in the neostriatum (Hallett, 1993). The principal neuron of the neostriatum, the medium spiny cell, has a distinctive pattern of firing, with bursts of spikes arising from noisy plateaus of depolarization ( 60 to 50 mv) lasting up to several seconds (Wilson, 1993). Between these plateau depolarizations the membrane potential of the cell is relatively stable and very hyperpolarized to near the reversal potential for potassium, about 90 mv. Using published voltage clamp data on the primary ion channels found in this cell type (Akins et al., 1990; Kitai & Surmeier, 1993; Nisenbaum & Wilson, 1995; Nisenbaum et al., 1994; Surmeier et al., 1988, 1989, 1991, 1992; Surmeier & Kitai, 1993), I have constructed a biophysically-based simulation model of a neostriatal medium spiny neuron, which reproduces this pattern of dynamically bistable membrane potential. The model includes modulatory influences of both dopamine and acetylcholine, and demonstrates how these modulatory neurotransmitters can interact to allow each medium spiny neuron to act as a one-bit short-term memory. I propose that the neostriatum is part of an architecture for sequential action which includes several such networks as components, and I have constructed a series of network models, which implement variations of the basic architecture and demonstrate that it is capable of controlling action sequences. The modulatory neurotransmitter dopamine plays a critical role in the functioning of the network models, as it does in the biological system. When simulated dopamine levels are lowered, the network models fail in ways which reproduce some experimental results which have been obtained using Parkinson s disease patients as subjects (Robertson & Flowers, 1990).
Chapter 2 Related work There have been a number of other computational models of the basal ganglia in recent years, focusing on various aspects of this very complex portion of the brain. Most of these are covered in the recent book Models of Information Processing in the Basal Ganglia (Houk et al., 1995). This chapter will focus on how other models relate specifically to the work described in this dissertation. 2.1 Abstract connectionist models 2.1.1 Berns & Sejnowski This rather complicated model uses the time delay of the additional synaptic connection in the indirect pathway through the GPe to implement a winner-take-all scheme for action selection (Berns & Sejnowski, 1994). It combines this with dopaminemediated reinforcement learning 1. Berns and Sejnowski s model of action selection is very different from the current proposal, but not necessarily in conflict with it. The claim that dopamine mediates reinforcement learning is plausible, given a change in the meaning of the dopaminergic signal from reward to significance. The Berns & Sejnowski model requires that the patch component of the striatum compute a temporal difference (Tesauro, 1992). It was not specified how this computation would be carried out. 1. The winner-take-all circuit in the Berns & Sejnowski model does not appear to require mutual inhibition among the medium spiny cells, a feature which has, in the light of recent results (Jaeger et al., 1994), become the Achilles heel of most models of the striatum. 5
6 2.1.2 Borrett et al. This model uses a recurrent connectionist architecture similar to a Jordan net (Jordan, 1985) with two hidden layers to learn a moderately complex pattern of oscillations (Borrett et al., 1993). The basal ganglia are identified with one of the hidden layers, and a decrease in gain on the input to the basal ganglia layer is shown to cause the oscillatory pattern to die away. This network may indeed capture some of what happens in Parkinson s disease, but its abstractness makes it difficult to map parts of the model onto parts of the brain. The issue of context or motor set is not raised. 2.1.3 Brotchie et al. This model identifies the globus pallidus with the middle layer of a threelayer recurrent net (Brotchie et al., 1991). It is shown that the activity of the middle layer resembles that of the globus pallidus during the execution of sequences. The issue of context or set is addressed by identifying the context units in the net with cortex. The parallels between hidden layer activation and activity in the pallidum are interesting, but once again, the extreme abstractness of the model makes it difficult to evaluate to what extent it applies specifically to the circuitry of the neostriatum. 2.1.4 Cohen Cohen et al. focus on the function of dopamine rather than on the basal ganglia, and most of their claims relate to schizophrenia and prefrontal cortex rather than to basal ganglia disorders (Cohen et al., 1992, 1996; Cohen & Servan-Schreiber, 1992, 1993; Servan-Schreiber et al., 1990). Cohen s conception of the function of dopamine has changed considerably in recent years. It was originally proposed that dopamine con-
7 trols gain on the inputs to prefrontal cortical areas, but this idea has now been abandoned in favor of the idea of gating. In this conception, dopamine acts as a signal which allows working memory to be loaded with new information, which then holds the information for as long as needed. During the hold time, external input to the cortical areas mediating working memory does not affect its contents. Cohen s models are abstract connectionist models rather than biophysical models, so again a more direct comparison to striatal circuitry is not possible, but the gating hypothesis is essentially compatible with the function of dopamine set forth in this dissertation, and Cohen s more recent models have components that function in a manner very similar to the neostriatum as implemented by my model. 2.1.5 Dominey & Arbib Dominey and Arbib s model (Arbib & Dominey, 1995; Dominey & Arbib, 1992) is very large and complex. It is implemented at a level of abstraction between connectionist networks and compartmental models, and is composed of layers which represent populations of neurons, with activation represented as continuous scalar quantities. The focus of the model is on generation of saccades, including memory-based saccades such as those studied by Hikosaka (1989). This model has gone through at least two stages of development. In the original version described in Dominey (1992), the caudate nucleus and substantia nigra pars reticulata are represented simply as delays in the pathway from the frontal eye fields (FEF) to the superior colliculus. The version described in (Arbib & Dominey, 1995) adds dopamine-mediated reinforcement learning, and modifies the role of the caudate to be a bias signal on the response of the superior
8 colliculus. This change moves the Dominey and Arbib model much closer to the architectures described in this dissertation, in sharp contrast with the previous version in which the striatum had no role in transforming information, but acted simply as a passive short-term buffer. 2.1.6 Jamieson Unlike most of the other models described here, Jamieson s (1991) does not use a recurrent net. Instead, Jamieson trained a variety of feedforward architectures to reproduce patterns of response to different doses of L-DOPA for different classes of Parkinson s patients. The model was tested by modifying various internal connections of each architecture after training, and comparing the results with clinical measures of motor performance or in this case, motor deterioration. Jamieson s argument is complex, but his main result is that an adequate model of the course of Parkinson s disease must incorporate differing degrees of striatal deterioration between different striatal areas. Jamieson s general method could conceivably be applied to my proposed model. 2.1.7 Mitchell et al. Mitchell et al. (1991) propose an approach to modeling the basal ganglia based on a view of the striatum as a pattern associator. Dopamine sets the thresholds of units in the striatum, allowing it to respond to wider or narrower sets of input patterns. This conception of the action of dopamine is similar to the gain control concept implemented in Cohen s earlier models. The suggestion is made that the striatum might play some role in encoding context in an arrangement similar to a simple recurrent network
9 (SRN) such as those studied by Jordan (Jordan, 1986) and Elman (Elman, 1990, 1991), but it is not clear if any of the ideas discussed in this paper have actually been put into the form of an actual simulation. 2.2 Biophysically-based models 2.2.1 Connolly & Burns Connolly & Burns (1993a, 1993b) model is not readily comparable to other biophysically-based models which use standard models of neurons as compartments and conductances, nor is it a connectionist model of any common type. Instead, Connolly and Burns propose that the striatum maps physical space into state space, and then programs actions efficiently by finding shortest paths through state space. It does this by functioning as a resistive network, by means of electrotonic connections between spiny cells. Although the idea of using the sort of map they propose to program actions is appealing, there is simply no evidence for the extensive electrotonic connectivity that Connolly & Burns model requires. 2.2.2 Porenta Porenta (1986) presents a model of a three-neuron loop consisting of dopaminergic, cholinergic, and GABAergic cells, with each cell synapsing on the next in serial fashion. He finds that a change in parameters to simulate Parkinson s disease (turning down the strength of the dopaminergic synapse) causes the system to have a shorter time constant. The implications of this shift are unclear. Porenta appears to have used parameters for nicotinic acetylcholine receptors in his model, but cholinergic activity in the striatum appears to be almost entirely mediated by muscarinic receptors, which have
10 very different dynamics from nicotinic receptors. It also appears that this model treats dopamine as a simple inhibitory neurotransmitter, almost identical to GABA except for its time constants. Given these considerations, it seems unlikely that the dynamics of this model are very close to those of the real system. 2.2.3 Wickens Wickens has implemented a number of models of the neostriatum, of varying degrees of complexity (Alexander & Wickens, 1993; Wickens, 1993; Wickens & Kotter, 1995; Wickens et al., 1991; Wickens & Arbuthnott, 1993). Although Wickens view of the function of the striatum in the overall scheme of things is very close to my own, his models all depend heavily on a striatal dynamics in which mutual inhibition plays a prominent role. Such a dynamics is difficult to reconcile with the very low firing rates of medium spiny neurons. The recent failure to detect any inhibitory influence between nearby spiny neurons (Jaeger et al., 1994) is a further problem for any model based on mutual inhibition. 2.2.4 Wilson Wilson has implemented a detailed biophysical model of the effects of potassium conductances on the cable properties of the dendritic tree of the striatal medium spiny neuron (Wilson, 1995). This model is to a large degree complementary to those described in this dissertation, in that Wilson focuses only on properties of the dendrites, and does not attempt a compartmental model of a whole medium spiny neuron. The model includes Hodgkin-Huxley models of both transient and non-inactivating potassium currents. The main result is that potassium currents produce nonlinearities in the
11 dendrites of medium spiny neurons which cause shunting of temporally and spatially random inputs, but allow precisely timed and spatially focused excitations to cause depolarizations. 2.2.5 Woodward et al. The model of Woodward et al. (Woodward et al., 1995) is conceptually similar to the model presented in this dissertation, in that it uses individual medium spiny neurons as elements of a working memory, and takes advantage of nonlinearities in the membrane properties in these cells to use the Up state to store information. It differs from the model presented here in that it does not include any modulatory neurotransmitters. Also, the Up state in the Woodward model serves only to enhance the stability of a working memory, in which an ON state is represented by continuous firing, and an OFF state is represented by quiescence. This is quite different from the model presented in this dissertation, in which subthreshold membrane potential is used to store information, and a cell can represent an ON state without generating action potentials. Another aspect in which the Woodward model differs from the present one is that the overall dynamics of the intrinsic circuitry of the neostriatum are mediated by reciprocal inhibition between medium spiny neurons. As noted above, there is currently no unequivocal evidence that such reciprocal inhibition exists, and the model presented here does not make use of it.
12 2.3 Summary Other than that of Porenta (Porenta, 1986), which assumes that dopamine is a straightforward inhibitory neurotransmitter, I was unable to find any biophysically-based models of the action of dopamine in the literature even Wickens (1993) model of the striatum omits dopamine from the picture. Considering the complex, often contradictory accounts of the effects of dopamine that one finds, this is understandable, but Kitai and Surmeier s recent research provides a good starting point for such a model (Kitai & Surmeier, 1993; Surmeier & Kitai, 1993). Validation remains a critically important issue for any model, whether abstract or detailed. Wickens manages to reproduce such classic deficits as parkinsonian performance on the Wisconsin Card Sort Test, despite the fact that his model is based on massive mutual inhibition, a phenomenon whose existence in the striatum is doubtful (Jaeger et al., 1994). This could mean that Wickens model achieves its results in an unrealistic manner, or it could mean that the experimental results indicating a lack of surround inhibition in the neostriatum are flawed. In general, detailed models are easier to evaluate, because every detail is a point that could potentially be checked against physiological data. With connectionist models, the mapping between model and brain is often so abstract that it becomes difficult to judge the validity of the model. Saying that lowering the strength of some connection in a model causes slowed responses, and then claiming that the model is therefore a model of bradykinesia, is really saying very little, because there are numerous architectures for which such an effect would be expected. This is not to say that abstract model cannot be of value. Cohen (1996), Mitchell (1991), and Berns and Sejnowski model
13 (Berns & Sejnowski, 1994) have all implemented abstract connectionist models in which the mapping between structures in the models and structures in the brain are specific enough to allow the models to be tested against physiological and behavioral data. One way to constrain a model enough to keep it true to the mechanisms in the biological system is to attempt broad coverage in the model s predictions. A valid biophysical model should be able to make predictions at many levels, from cellular physiology to behavior, reproducing the dynamics of the system at intermediate levels of description as well. To my knowledge, no computational model of the basal ganglia has managed to cover all of these levels at once. In the models presented here, I have started with low-level physiology, and have attempted to remain faithful to that physiology while reproducing low-level network dynamics as well as high-level behavioral phenomena. The models presented here cover phenomena ranging from membrane potential transitions to behavior, and in so doing cover the widest stretch of any that I know of.
Chapter 3 Background 3.1 Nuclei of the basal ganglia Because the nomenclature of the basal ganglia has changed somewhat in recent years, it is important to be clear about exactly what set of structures are being considered here. Older accounts tend to follow physical boundaries, and refer to the group of forebrain structures comprised of the striatum, globus pallidus, and claustrum as the basal ganglia. This set of nuclei was referred to by some authors as the corpus striatum (Carpenter, 1991), but this term term has now largely fallen into disuse (Parent, 1996). The substantia nigra was grouped with other midbrain structures, and the subthalamic nucleus with the thalamus. On the basis of their functional interrelationship and mutual connectivity, almost all recent accounts are explicit in including the caudate nucleus, putamen, globus pallidus, substantia nigra, and subthalamic nucleus in the basal ganglia (Côté & Crutcher, 1991; Parent, 1986, 1996; Wilson, 1990). Although the claustrum is included as part of the striatum due to its location, developmental origin, and histological similarity to other striatal structures, it does not appear to participate in any of the major circuits controlling action selection, and is therefore usually omitted from consideration in recent accounts. The nucleus accumbens is a structure lying medial and ventral to the caudate and putamen, often referred to as the ventral striatum, whose connectivity and internal structure justify its inclusion with the other striatal structures. Finally, the ventral tegmental area (VTA) is a structure lying 14
15 adjacent to the dopaminergic pars compacta of the substantia nigra. Because the VTA is also largely dopaminergic, it is sometimes grouped with the substantia nigra, but its outputs go directly to prefrontal cortex, not to the striatum. In the following text, I follow Parent (1996), and use the term basal ganglia in the broad sense to refer to the caudate, putamen, nucleus accumbens, globus pallidus, subthalamic nucleus, and substantia nigra. 3.2 Anatomy of the basal ganglia Figure 1 shows the major nuclei of the neostriatum, the caudate and the putamen, as they are physically situated in relation to the neocortex. As can be seen from the figure, they are rather large structures, collectively the second largest forebrain structure striatum: caudate putamen globus pallidus FIGURE 1. Large structures of the basal ganglia in relation to the cortex
16 after the cortex itself. Although sometimes discussed separately, the primate caudate and putamen are physically continuous, with the internal capsule coursing through the middle of the composite structure. This interruption is not complete, and filaments of striatal tissue join the two structures in areas surrounding the internal capsule. The internal structure of the caudate and putamen appear identical. In the rat, the internal capsule breaks up into numerous small bundles as it passes through the striatum, leaving a unified nucleus usually referred to as the caudoputamen (Parent, 1986; Wilson, 1990). There are differences in external connectivity between the caudate and putamen, but I believe that this reflects a continuous gradient across the entire neostriatum, rather than any sharply delineated physical or functional boundary between the caudate and the putamen. The nucleus accumbens appears to continue the connectivity gradient which runs through the other two striatal structures. For the remainder of this paper I will refer to all three collectively as the neostriatum or simply the striatum. Figure 2 gives an overview of the connectivity of the basal ganglia, summarized in Figure 3. Almost all areas of the neocortex project to the striatum (Graybiel & Ragsdale, 1979). The strengths of these projections are not uniform. Primary visual cortex, for instance, is generally considered to have only a very weak projection to the striatum this dissertation (Kemp & Powell, 1970) (though it does in fact project strongly to the claustrum). Projections from the basal ganglia back to the cortex are more restricted. Alexander, DeLong, & Strick (1986) identified 5 major loops through the basal ganglia, as shown in Figure 4, terminating in anterior cingulate area (ACA), lateral orbitofrontal cortex (LOF), dorsolateral prefrontal cortex (DLC), frontal eye fields (FEF), and supple-
17 mentary motor area (SMA) which they note is not to be taken as an exhaustive list of basal ganglia targets. Based on studies using both anterograde and retrograde tracers, there appears to little or no overlap between these circuits, leading to the notion that the neocortex IV V VI basal ganglia telencephalon patch matrix striatum globus pallidus GPe GPi diencephalon STN thalamus VAmc VLm MDpl RN VApc VLo CM mesencephalon substantia nigra SNc SNr Abbreviations: globus pallidus: GPe: external segment GPi: internal segment Neurotransmitters: glutamate GABA dopamine substantia nigra: SNc: pars compacta SNr: pars reticulata STN: subthalamic nucleus thalamus: see Figure 4 FIGURE 2. Block diagram of the basal ganglia and related structures
18 neocortex basal ganglia striatum globus pallidus substantia nigra subthalamic nucleus Neurotransmitters: glutamate GABA dopamine thalamus FIGURE 3. Summary of basal ganglia connections basal ganglia are divided into small functional units acting independently of each other. 3.2.1 The neostriatum The great majority of input to the basal ganglia comes by way of the striatum. There is a topographic organization to striatal input, in all three dimensions. As shown in Figure 5, there are gradients in terms of body map (Flaherty & Graybiel, 1991), areas of origin of afferent fibers (Schultz et al., 1992; Selemon & Goldman-Rakic, 1985), and specificity of input (Flaherty & Graybiel, 1991, 1993). The last of these bears explanation: at the extreme caudal end of the striatum, cortical inputs from areas representing particular body parts tend to overlap very little with adjacent representations. So, for
19 APA, MC, SC DLC, PPC PPC, APA STG, ITG, HC, EC, ACA STG, ITG Cortex SMA FEF DLC LOF ACA Striatum PUT CAUD (b) dl CAUD (h) vm CAUD (h) VS Pallidum/ S. Nigra vl GPi cl SNr cdm GPi rl SNr ldm GPi rm SNr mdm GPi rm SNr rl GPi, VP rd SNr Thalamus VLo VLm l VAmc MDpl VApc MDpc m VAmc MDmc pm MD FIGURE 4. Loops through the basal ganglia, from Alexander, DeLong, & Strick(1986). Abbreviations: ACA: anterior cingulate area; APA: arcuate premotor area; CAUD: caudate, (b) body, (h) head; DLC: dorsolateral prefrontal cortex; EC: entorhinal cortex; FEF: frontal eye fields; GPi: globus pallidus internal segment; HC: hippocampal cortex; ITG: inferior temporal gyrus; LOF: lateral orbitofrontal cortex; MC: motor cortex; MDpl: medialis dorsalis pars paralamellaris; MDmc: medialis dorsalis pars magnocellularis; MDpc: medialis dorsalis pars parvocellularis; PPC: posterior parietal cortex; PUT: putamen; RN: reticular nucleus; SC: somatosensory cortex; SMA: supplementary motor area; SNr: substantia nigra pars reticularis; STG: superior temporal gyrus; VAmc: ventralis anterior pars magnocellularis; VApc: ventralis anterior pars parvocellularis; VLm: ventralis lateralis pars medialis; VLo: ventralis lateralis pars oralis; VP: ventral pallidum; VS: ventral striatum; cl: caudolateral; cdm: caudal dorsomedial; dl: dorsolateral; ldm: lateral dorsomedial; m: medial; mdm: medial dorsomedial; pm: posteromedial; rd: rostrodorsal; rl: rostrolateral; vm: ventromedial; vl: ventrolateral. instance, the striatal representations of two adjacent fingers are completely separate at the caudal end of the striatum. Moving toward the rostral end of the structure, body part representations overlap more and more, until at the extreme rostral end of the striatum, it is impossible to identify portions of striatum as representing particular body parts at all. Almost all areas of the cortex project to the neostriatum, and functionallyrelated cortical areas sometimes converge on single small areas of the neostriatum (Parthasarathy et al., 1992). This pattern has led to the suggestion that individual striatal
20 limbic sensorimotor caudate interdigitate overlap putamen foot head FIGURE 5. Topographic organization of striatal input (from (Nieuwenhuys et al., 1981)) medium spiny neurons are acting as conjunction detectors which fire when meaningful combinations of input are simultaneously active, and the properties of the dendrites of medium spiny neurons have led to similar suggestions on physiological grounds alone (Wilson, 1992, 1995). The topography of striatal inputs and internal structure, as described above, are also ideally suited for covering the space of meaningful conjunctions.
21 caudate putamen A FIGURE 6. B Cytoarchitecture of the neostriatum a) caudate and putamen, showing typical mix of small and large cell bodies b) typical patch/matrix pattern superimposed There are also glutamatergic inputs to the striatum from the thalamus, serotonergic inputs from the dorsal raphe nucleus, and inputs from the amygdala which are assumed to be excitatory (Côté & Crutcher, 1991; Parent, 1996; Wilson, 1990). The cytoarchitecture of the striatum appears uniform throughout, with numerous small cells dotted by occasional much larger ones, as shown in Figure 6. Techniques which reveal histochemical variations give a very different picture. First discovered in 1978 with staining for acetylcholinesterase (Graybiel & Ragsdale, 1978a, 1978b), the original grouping of the striatum into striosome (or patch ) and matrix compartments has now been elaborated into a view in which the striatal matrix is a collection of matrisomes of about the same size as the original patches (Graybiel et al., 1991; Malach & Graybiel, 1986, 1987). In addition, even more recent studies have iden-
22 tified other striatal inhomogeneities in terms of the distributions of various neuropeptides (Groves et al., 1995; Martone et al., 1992), which show complex patterns of interrelationship with the striosomes. Cortical projections to the neostriatum are segregated by both the original patch/matrix compartments and by matrisome boundaries. There does seem to be some special significance to the original patches, however. In addition to differences in corticostriatal projections to the patches (Giménez-Amaya & Graybiel, 1990, 1991), projections to the striatum from the dopaminergic portion of the substantia nigra also respect patch boundaries (Langer & Graybiel, 1989). In the cat, corticostriatal projections show a particularly intriguing pattern relative to the patch and matrix compartments: several cortical areas 1 which project to the matrix compartment at the more limbic pole of their corticostriatal projection, project to the patch compartment at the more sensorimotor end of the projection (Ragsdale & Graybiel, 1990). No exceptions to this pattern were found. Although Ragsdale and Graybiel declined to speculate on the functional significance of this regularity, the cortical areas involved have some features in common: all are considered association cortex, and all appear to be involved in some way with attention. Although at this time there is only one study available which concentrates on this pattern, examination of other published results tracing corticostriatal connections suggests similar patterns in the monkey (Alexander et al., 1986; Selemon & Goldman-Rakic, 1985) and rat (Kincaid & Wilson, 1996). This pattern of innervation is consistent with my proposal of a hierarchical organization, which is furthermore a hierarchy of atten- 1. Cortical areas cited by Ragsdale and Graybiel (1990) include posterior parietal, dorsomedial prefrontal, ventrolateral prefrontal, insular, and rostral temporal cortices.
23 tional mechanisms. There is evidence that attentional deficits are part of the symptom complex of Parkinson s disease (Brown & Marsden, 1988a; Ljungberg et al., 1991), and difficulty in concentration or obvious lapses in attention are often the presenting symptom in Huntington s disease (Hayden, 1981; Klein, 1981). Output from the neostriatum goes primarily to four nuclei: the internal and external segments of the globus pallidus (GPi and GPe), and both the pars compacta and pars reticulata of the substantia nigra (SNc and SNr) (Graybiel & Ragsdale, 1979). The GPi and SNr are the output structures of the basal ganglia as a whole, while the SNc provides dopaminergic input back to the striatum. The precise nature of the striatal projection to the SNc is somewhat unclear, since the striatal neurons projecting to the SNc express both GABA, which is inhibitory in most contexts, and substance P, a peptide which is generally considered excitatory (Bolam & Izzo, 1988; Stanfield et al., 1985). Some authors have proposed that the striatal inputs to the SNc may actually act in a net excitatory manner by inhibiting glutamatergic inputs to the dopaminergic cells (Grace, 1993). However, there is evidence that cell loss in Huntington s disease may begin with projections from striatal patches to dopaminergic cells, resulting in dopaminergic hyperactivity, which may be the cause of the choreic movements which are a prominent symptom of Huntington s disease (Hedreen & Folstein, 1995). If so, this would imply that striatal projections to dopaminergic cells are inhibitory. Although it seems unlikely that there is an undiscovered corticostriatal link that would contradict the idea of functionally segregated subunits in the striatum, there is still the possibility of some sort of indirect linkage by way of the SNc. Selemon and
24 Goldman-Rakic (1985) and others have found that projections from the cortical areas involved in Alexander, DeLong & Strick s parallel loops interdigitate in the striatum in complex ways, but for the most part without much overlap. Some of these interdigitations have been found to lie along patch-matrix boundaries, and the patch and matrix compartments have different output targets as well as different inputs (Gerfen, 1989; Jimenez-Castellanos & Graybiel, 1989b; Ragsdale & Graybiel, 1988, 1990, 1991). The proposed source of the linkage between these interdigitated connections is in the dopaminergic nigrostriatal projections. Although current tracing techniques are not sufficiently precise to demonstrate conclusively that particular dopaminergic cell project back to the matrix surrounding the patches which were the source of their input, the available evidence is consistent with this hypothesis (Langer & Graybiel, 1989). Thus, the patches could indirectly provide dopaminergic innervation to the surrounding matrix. The picture presented so far is that there is a topographic gradient in the striatum in terms of cortical areas, proceeding from limbic, through prefrontal, to sensorimotor. These cortical areas participate in closed loops through the basal ganglia, which appear independent due to the interdigitation of their destinations in the striatum. Available evidence suggests, however, that there is a link spanning these interdigitations, by way of the substantia nigra pars compacta. If, as is likely the case, the patches exert dopaminergic influence on the surrounding matrix, this means that more limbic cortical areas are capable of influencing prefrontal and sensorimotor cortical areas by way of the striatum.
25 3.2.2 The internal segment of the globus pallidus (GPi) and pars reticulata of the substantia nigra (SNr) The GPi and SNr are the main output structures of the basal ganglia. Both project to the thalamus, with the SNr projecting also to the superior colliculus and midbrain tegmentum. All output from the GPi and SNr appears to be GABAergic. In contrast to the quiet cells of the striatum, the cells of these nuclei are tonically active, with average firing rates of around 50 100 Hz (Hikosaka & Wurtz, 1983b). There is no single characteristic which differentiates pallidal and nigral targets in the thalamus. Both project to the VA-VL complex, and although there appears to be little or no overlap between pallidothalamic and nigrothalamic fibers (Graybiel & Ragsdale, 1979), both GPi and SNr participate in loops involving limbic, prefrontal, and sensorimotor cortical areas see Figure 4, page 19 for examples. The SNr, through its connection to the superior colliculus, is also able to exercise strong control over the time of execution and target location of saccadic eye movements (Hikosaka & Wurtz, 1983a). In the cat, the SNr projects also to the reticular nucleus of the thalamus (Pare et al., 1990). The striatonigral and striatopallidal projections exhibit a complex structure. For the most part these details will not be dealt with here, but one regularity in particular is worth noting: projections from the cortex to different parts of the striatum frequently reconverge, projecting via the striatum to single targets in the GPi. This pattern probably occurs in the substantia nigra as well (Flaherty & Graybiel, 1994).
26 3.2.3 The external segment of the globus pallidus (GPe) The GPe receives inhibitory input from the striatum and the GPi, and excitatory input from the STN (Giménez-Amaya & Graybiel, 1990; Hazrati et al., 1990). Its output is entirely GABAergic, and is directed mainly at the STN. There is also evidence for a projection from the GPe to the reticular nucleus of the thalamus (Hazrati & Parent, 1991). Together with the previously noted projection from the SNr, this projection could provide a pathway for the basal ganglia to exert broad control over much of the cortex. 3.2.4 The pars compacta of the substantia nigra (SNc) The SNc and an adjacent nucleus, the ventral tegmental area (VTA) together provide most of the dopamine in the entire brain. It is loss of SNc neurons which leads to the striatal dysfunction which directly causes Parkinson s disease. The SNc receives tonic excitation via a glutamatergic projection from the STN, and GABA- and substance P-ergic input from the striatum. Projections from the striatum to the SNc are topographically organized, but are not continuous with those to the adjacent SNr (Hedreen & DeLong, 1991). 3.2.5 The subthalamic nucleus (STN) The projection cells of the STN are glutamatergic, making this small nucleus the only one of the basal ganglia whose main output is excitatory (Graybiel & Ragsdale, 1979). In contrast to the precisely organized projections between other nuclei of the basal ganglia, that from the STN to the GPi and SN appears relatively diffuse. Thus, output from the STN appears to exert a veto on striatal output pathways, by opposing the GABAergic striatal input to the GPi and SNr. The exact scope of a single projection in