CREATING FUNCTIONAL NEURAL CONTROL CIRCUITS

Transcription

1 CREATING FUNCTIONAL NEURAL CONTROL CIRCUITS INCORPORATING BOTH DISCRETE TIME, MAP BASED NEURONS AND HINDMARSH-ROSE ELECTRONIC NEURONS AUTHOR: DANIEL KNUDSEN, BNS 2006 ADVISOR: DR. JOSEPH AYERS COLLEGE OF ARTS & SCIENCES JR/SR HONORS PROJECT SUBMITTED: SPRING, 2006

2 TABLE OF CONTENTS ABSTRACT... 3 SYNTHETIC NEUROSCIENCE... 3 TYPES OF CENTRAL PATTERN GENERATORS... 5 ENDOGENOUS BURSTER... 5 RECIPROCAL HALF CENTER... 6 RECURRENT CYCLIC INHIBITION... 6 UCSD ANALOG ELECTRONIC NEURONS... 7 THE UCSD HIND MARSH-ROSE ELECTRONIC NEURON EQUATIONS... 7 ELECTRONIC CHEMICAL SYNAPSE EQUATIONS... 8 BUILDING BASIC CPGs WITH ELECTRONIC NEURONS DISCRETE-TIME MAP-BASED NEURONS RULKOV TWO-DIMENSIONAL MAP EQUATIONS SYNAPSE EQUATIONS BUILDING BASIC CPGs WITH MAP-BASED NEURONS COMMAND NEURON, COORDINATING NEURON, CPG ARCHITECTURE COMPLEX NETWORKS BUILT WITH MBNs: COMMAND AND COORDINATION HYBRID NETWORKS: INTERFACING ANALOG ELECTRONIC NEURONS WITH DIGITAL MAP-BASED NEURONS WHY A HYBRID ARCHITECTURE? MAP-BASED NEURONS SYNAPSING ONTO ELECTRONIC NEURONS ELECTRONIC NEURONS SYNAPSING ONTO MAP-BASED NEURONS TWO-WAY SYNAPTIC ACTIVITY DISCUSSION REFERENCES

3 ABSTRACT Unit central pattern generators (CPGs) were created using two different types of modeled neurons. Three basic central pattern-generating networks were created for each of two separate types of modeled neurons. First, UCSD analog electronic neurons (ENs) based on the three Hindmarsh-Rose differential equations, plus a fourth differential equation developed at UCSD, were used to build an endogenous pacemaker network, a reciprocal half center network, and a recurrent cyclic inhibitory network. These electronic neurons are a biologically realistic phenomenological model, but are computationally intensive and difficult to prototype with. The three CPG types were then also created using discrete-time map-based neurons (MBN s ), which are computationally less intensive, and also less biologically accurate. Finally, networks were created using both artificial neuron types, showing the possibility for creating hybrid circuits for the control of robotics or neural prosthetics that can benefit from the biological realism of the ENs and the computational efficiency of the MBN s. SYNTHETIC NEUROSCIENCE The animal kingdom shows an incredible diversity of adaptive behaviors that help individuals interact with a dynamic and inconsistent world. Almost never does a walking animal get a uniformly flat, hard surface upon which to walk. It encounters and must overcome obstacles, changes in pitch, and different surfaces which allow for greater or less friction. The neural control of these complex adaptive behaviors is far from being understood entirely, but progress is being made. With new knowledge in this area comes the potential for treating neural pathologies, better understanding biological control systems, and learning how to create more realistic biomimetic devices. Understanding the neural circuitry that gives rise to behavior can explain the reasons for failure in normal behavior, as in disease or injury. It can also shed light on general techniques for the control of a complex neural arrangement as exists in almost all animals and certainly in humans. Since extracting data from biological circuits is technically very challenging, and actually constructing certain desired neural circuits in a biological medium is virtually impossible, the technique of computer modeling has proved to be a very powerful tool in studying proposed connections and behaviors of neural circuitry. Since Hodgkin and Huxley first described the action potential in terms of differential equations representing the changing ionic currents across the cell membrane in 1952 (Hodgkin & Huxley 1952), modeling neural processes has been an important tool in researching possible mechanisms of network action. Another promising avenue for gaining a better understanding of neural circuits that takes the concept of neuronal network modeling one step further is the idea of synthetic neuroscience. This is a relatively new field with the goal of building engineered devices that incorporate neurobiological control principles. These devices may be biomimetic robotics, which could lead to advances in the understanding of the organization and implementation of neurobiological control systems, or they may be neuroprosthetic devices, used to aid patients with disorders of 3

4 the nervous system. Synthetic neuroscience requires that the device be able to obtain information from its environment, process this information in real-time, and output data or movement in the form of some sort of actuation. Synthetic neuroscience has already seen many successful implementations, such as the construction of robot models of animals with control systems based on the principles of neurophysiology. Biomimetic robots have been created to model many different organisms from insects to fishes to crustaceans. Dr. Joseph Ayers' robotic lobster recreates the locomotor motor program and behavioral hierarchy of the American lobster, Homarus americanus. The current version of the robotic lobster is controlled by a finite state machine phenomenologically based on the behavioral hierarchies and locomotor patterns seen in biological lobsters (Ayers 2002, Ayers et a!. 2005a,b). While the current robotic lobster successfully recreates the range of behaviors seen in biological lobsters, it is thought that a control system more directly based on biological principles would allow for an even greater breadth and depth of autonomous adaptive behaviors. This would also allow for the direct incorporation of control mechanisms seen in biological systems such as command modulation, pattern entrainment, as well as emergent properties of networks difficult to model using a state-machine architecture. With these considerations in mind, it was the purpose of this research to create technology that will allow the next generation of the robotic lobster built by Dr. Ayers' group to incorporate many more adaptive behaviors than previous versions (Ayers 2002). This was accomplished by using two different types of modeled neurons, computed discrete-time, map based neurons (MBN s, Rulkov 2002) and electronic neurons instantiating the Hindmarsh-Rose equations (Pinto 2000). These circuit media, discussed in detail below, will enable these adaptive behaviors through a hybrid architecture that is a combination of the MBN s and ENs, as described above. The advantage to this approach is that the MBN s are computed quickly and efficiently and lend to the incorporation and communication of sensory and command information. Though the ENs are more biologically accurate, they require too much circuit board real estate to allow the type of neural complexity needed in the command system (Ayers et a!. 2005a). The hybrid control system will allow both exteroceptive, that is sensory information from the environment, and proprioceptive, that is sensory information from within an organism that allows for reflexes, information to be combined in a timely and effective manner, giving a robot with this type of control system a much better ability to adapt its behavior in a given situation. This type of animal-like adaptability is much sought after in the design of autonomous robots, as it is impossible to write a control program that can account for every possible situation a robot might encounter. The combination of the two different neuron types will allow a robust implementation of the command neuron-coordinating neuron-cpg architecture as described in Stein (1978) and Ayers et a!. (2005a,b), and lead to a more biologically accurate biomimetic lobster. Three basic central pattern generator types were created with both MBN s and ENs. The three different elementary CPG mechanisms that were created were: endogenous bursters, reciprocal inhibitory networks, and networks mediated by recurrent cyclic inhibition (Selverston 1980). 4

5 The two circuit media were then integrated into a hybrid architecture, showing that functional circuits may be constructed from a combination of the different types of neurons. The main significance of this research is the reduction in computational energy required to integrate sensory information into adaptive motor patterns, while still allowing networks to work off of biologically realistic principles at the neuronal level. It also sets precedence excitatory synapse e inhibitory synapse A: Endogenous Burster EB inhibits follower F B: Endogenous Burster EB excites follower F C: Reciprocal Half Center CPG, activated by command neuron cmd D: Recurrent Cyclic Inhibition CPG, activated by command neuron cmd Figure 1: Basic Central Pattern Generator types. for the digital to analog and analog to digital communication between MBNs and ENs. TYPES OF CENTRAL PATTERN GENERATORS ENDOGENOUS BURSTER The endogenous burster central pattern generator is perhaps one of the simplest, in that it may consist of a single neuron. This neuron possesses channels that imbue it with a pacemaker-like activity, and for this reason endogenous burster CPGs are also often called endogenous pacemaker CPGs. Although the network connectivity may be simple, the biophysical properties of neurons of this type are anything but simple. The action potential activity comes from the interaction of the normal channels: leak channels, sodium channels with inactivation characteristics, and potassium channels with delay characteristics, but the pacemaking character of these neurons is due to a potassium activated calcium channel. This channel causes a rhythmic depolarization of the cell due to an influx of calcium driven by potassium influx during normal repolarization. In general, one or more pacemaker cells will either excite or inhibit a number of other cells, which will then either fire in or out of phase with it. This arrangement has been found in many different biological networks. Endogenous burster networks have been implicated in circuits as diverse as the heartbeat of leech and Aplysia (Selverston 1980), the pyloric rhythm in the stomatogastric ganglion of crustaceans, the mammalian respiratory motor pattern (Marder and Calabrese 1996), and the walking locomotor program of the lobster (Ayers and Davis 1997), among many others. See Figure la,b for a diagrammatic representation of an endogenous pacemaker CPG. 5

6 RECIPROCAL HALF CENTER The reciprocal half center is a unit CPG comprised of two tonically spiking neurons connected via equal strength inhibitory synapses. While one neuron is firing, it inhibits the other from firing at all and continues to fire until some asymmetry such as synaptic depression or cell fatigue lets the other neuron overcome its inhibition. As this other neuron begins to fire, it inhibits the first and the cycle repeats. This arrangement leads to an oscillatory pattern, as the neurons fire antagonistically and exactly out of phase. This type of network has also been implicated in many different systems including swimming motor pattern generation in leech, lamprey, fishes and other animals, as well as in the leech heartbeat, and the gastric mill in the crustacean stomatogastric nervous system (Selverston 1980, Marder and Calabrese 1996, Marder 2000). See Figure 1C for a diagrammatic representation of a reciprocal half center CPG. An important mechanism underlying this and the recurrent cyclic inhibitory CPG, discussed below, is that of post inhibitory rebound (PIR). PIR is the tendency for a neuron to fire a burst of action potentials after it has been released from inhibition. This happens because as the neuron is hyperpolarized while under inhibition, the number of inactivated sodium channels decreases, and this lowers the firing threshold of the neuron. If the inhibition is released quickly, and the neuron repolarizes rapidly, its membrane potential may exceed the new, reduced threshold and send the neuron into a burst of spikes. If the repolarization is more gradual, the sodium channels may be given a chance to inactivate, and so the threshold may stay above the membrane potential and the neuron will not fire. Therefore, an important characteristics of networks based on PIR is that the inhibition to a postsynaptic cell is released quickly. RECURRENT CYCLIC INIDBITION Recurrent cyclic inhibition is caused by a closed loop of inhibitory synapses as was the reciprocal half center, but now there is an odd number of neurons in the circuit. PIR is again the main mechanism of action, with cells firing bursts of action potentials as they are released from inhibition by their presynaptic cell. The number of phases in the overall network activity will equal the number of cells that make up the inhibitory loop. This network has been suggested as the main mechanism of action in the leech locomotor rhythm (Friesen et a!. 1978), as well as in the intersegmental coordination of that system. See Figure 1D for a diagrammatic representation of a recurrent cyclic inhibition CPG. As and one cell begins to fire, it is doing so because the cell directly presynaptic to it has just finished firing and has released it from inhibition. As the current cell fires, it inhibits its postsynaptic target, and the cell previous to it remains in a non-spiking state. When the current cell stops firing, it causes a PIR-induced burst in its postsynaptic partner, which then inhibits its postsynaptic partner. The circuit continues around the loop like this until the cells undergo cellular or synaptic fatigue. The recurrent cyclic inhibitory network must be made of an odd number of cells, because if there are an even number of cells cyclically inhibiting each other, the network would eventually settle into a two-phase pattern, with every other neuron firing at the same time, exactly antagonistic to the neurons on either side of it. The overall network activity would end up resembling that of a two-phase reciprocally inhibitory network. 6

7 UCSD ANALOG ELECTRONIC NEURONS The electronic neurons (ENs) and electronic chemical synapses (ECSs) used in this study (see Fig. 2) were developed at the Institute for Nonlinear Science (INLS) at the University of California San Diego (UCSD) and are based on the Figure 2: The UCSD electronic neuron (top) and the UCSD electronic chemical synapse (right). Hindmarsh-Rose equations for either two- three- or four-dimensional neurons. A good description of these neurons and their capabilities can be found in Pinto et al The ENs are analog computers which instantiate these equations via analog components and can be configured to display tonic spiking and also bursting waveforms that very accurately reflect biological neurons. Although the original Hindmarsh Rose equations were three differential equations (Hindmarsh & Rose 1984), these ENs include an extra equation, dw(t)/dt that models slow calcium dynamics and has been shown to control the amount of chaotic behavior that gives rise to irregular spiking and bursting. Figure 3 shows the four equations that describe the circuitry on the ENs. These neurons may be connected to each other via electronic chemical synapses that can simulate the coupling found in biological synapses that gives rise to network activity. The equations instantiated in the ECSs can be seen in Fig. 4. The particular ENs and ECSs used in this study were fabricated in-house at the Ayers Robotics Laboratory, using commercially available analog circuit media. THE UCSD HINDMARSH-ROSE ELECTRONIC NEURON EQUATIONS As described above, the Hindmarsh-Rose Equations are a system of three differential equations (Eqns.l-3, below) that create a phenomenological model of spiking and bursting neurons. Eqn. 1 represents the transmembrane voltage of the EN, Eqn. 2 represents the fast dynamics that give rise to spiking, and Eqn. 3 represents the slow dynamics that give rise to bursting. Eqn. 4 was not part of the original Hindmarsh-Rose Equations, but provides a method for modeling slow calcium dynamics. These calcium dynamics were not studied in detail for the current report. 7

8 The Hindmarsh-Rose Equations dx(t)/dt = ay(t) + bx2(t)- cx3(t) -dz(t) +i (1) dy(t)/dt = e- fx2(t)- y(t)- gw(t) (2) dz(t)/dt = m(-z(t) + s(x(t) +h)) (3) dw(t)/dt = n( -kw(t) +r(y(t) + 1)) ( 4) Figure 3: (Eqns 1-3) The Hindmarsh-Rose Equations. (Eqn 4) models the effects of calcium currents. These are instantiated in the analog circuitry of the EN. Equation 1 is responsible for the transmembrane voltage, 2 represents fast dynamics, 3 slow dynamics, and 4 calcium dynamics. Equations 3 and/or 4 may be selectively turned off to produce either 2, 3, or 4-dimensional ENs. The variables i, e, g, m, h, n, and 1 can be adjusted by tuning variable resistors on the ENs. The ENs include switches that remove Eqns. 3 and/or 4 from the circuit, allowing quick switching between 2-dimensional spiking neurons, 3-dimensional spiking-bursting, or 4- dirnensional spiking-bursting ENs displaying calcium dynamics. Additionally, the variables i, e, g, m, h, n, and 1 can be varied by adjusting variable resistors on the EN hardware. All other variables are constant. ELECTRONIC CHEMICAL SYNAPSE EQUATIONS Below is the system of equations used to create the ECSs. Ic is added to the I variable in Eqn 1, of the postsynaptic EN, mimicking the influx of positive or negative current to the cell determined by the synapse. The variables Gc, Xrev, xth, and Xslo p e are controllable by variable resistors on the ECS hardware. Gc determines the strength of the synapse, Xrev defines the reversal potential and therefore determines whether the synapse will be inhibitory or excitatory, Xtb sets the presynaptic threshold voltage for the synapse, and Xslo p e sets the synaptic time constant. X p ost is the postsynaptic EN voltage (ie the value of Eqn 1), and X p re is the presynaptic EN voltage, and both are determined by the wiring of a given EN network. 8

9 Electronic Chemical Synapse Equations I c= 213G CS( t) (X rev-xpost) ds(t) (I - S oo) 1"s d t = (S 00 - S ( t)) Soo(Xpre) = tanh [X p re-xthl X slope (5) (6) (7) Figure 4: Electronic Chemical Synapse Equations. Figure from Pinto et al. 2000; Eqns 5, 6, and 7 here are eqns 4, 5, and 6 there, respectively. Ic is added to i of the postsynaptic EN. The variables Gc, Xrev x1h, and XsJope may be controlled on the ECS by tuning variable resistors. Fig. 5 shows waveforms generated by ENs with different values for the seven parameters in Fig. 3. The ENS display tonic spiking and bursting behaviors with various interspike intervals, interburst intervals and duty cycles. Panels G and H show ENs with two very different spikingbursting patterns after changing only one variable (e). See Pinto et al. 2000, for a more detailed description of the ENs and ECSs. g (ohms) 0 0 n(ohms) m(ohms) (V) i (V) 5.78 i (V) 5.78 e(v) 0 e(v) 1.18 h(v) h(v) G H Figure 5: Examples of waveforms created by ENs tuned to different parameters. Panels G and H show that very different neuron types can be obtained while changing only one variable; in this case, e. Here and in all future figures showing neuronal waveforms, the y axis is transmembrane voltage (the value of x in Eqn 1), and the y axis is iteration number (time). 9

10 BUILDING BASIC CPGS WITH ELECTRONIC NEURONS The ENs and ECSs described above were used to create the three basic CPG types. A synaptic connection was made by connecting the membrane voltage of the presynaptic cell (x in Eqn 1) to the terminal for Xpre on a chemical synapse. The membrane voltage of the postsynaptic cell to Xpost is then connected on the same synapse, and finally, connecting the Ic of that synapse to the current input of the postsynaptic cell (i in Eqn. 1). Recordings were made by connecting the output of the cells to a terminal block connected to an analog acquisition board made by National Instruments (NI PCI-6221). This board performed analog to digital conversion and displayed the output of the cells in National Instruments Labview 7.1 and 8.0, a device-independent signal acquisition, analysis and simulation graphical interface program. The sampling rate was 1000Hz for Figs Additionally, all figures in this document depicting neuronal waveforms, whether EN or MBN, have the iteration number of the acquisition program as their x-axis, this is analogous to time. The y-axis of these figures is the voltage of these neurons. The figures here are qualitative in nature, and as such, no strict relationship between iteration number and absolute time was attempted, nor was it attempted for voltage. ENDOGENOUS BURSTER Endogenous burster networks, as described above and depicted in Fig. la,b, were created with ENs and ECSs. Panel A of Fig. 6 shows the endogenous burster, EB, and the follower neuron, F, when no synapse is present. The EN parameter values, as well as the parameter values for the ECSs described below, are shown on the right. One may notice that EB is bursting intrinsically A M(oluns) Vrev (v) iterations= 6000 synapse introduced c Figure 6: A simple endogenous burster CPG made with ENs. EB is intrinsically active, while F is intrinsically silent. In A, there is no synapse between EB and F. In B, F begins to fire with EB after an excitatory synapse is introduced. InC, F fires out of phase with EB when an inhibitory synapse is introduced. The right panel shows the parameters associated with the networks. The y-axis is voltage, and the x-axis is iterations (time). 10

11 while F is silent. When an excitatory synapse from EB to F is introduced in B, F begins to fire in phase with EB. If an inhibitory synapse is introduced instead, as inc, F begins to fire out of phase with EB due to post inhibitory rebound. RECIPROCAL HALF CENTER ENs were also used to create a reciprocally inhibitory half center network. Figure 7 A shows the first network created, which was made with intrinsically spiking ENs connected via inhibitory synapses. When HC1 is firing, it completely inhibits HC2, and vice-versa. After a short interval, HC2 takes over and completely inhibits HC 1. They fire perfectly out of phase and oscillate indefinitely. A reciprocal half center network was then created in which HC1 and HC2 were intrinsically silent. When HC 1 was given a brief positive current injection, it began to fire and the network entered an oscillatory state, with bursts of each HC being caused by post inhibitory rebound. Figure 7: A simple reciprocal half center CPG made with ENs. In A, two intrinsically spiking neurons are connected via identical inhibitory synapses. B shows intrinsically silent spiking neurons that begin to show an oscillatory rhythm after a brief current injection to HCl. RECURRENT CYCLIC INHIBITION ENs were also used to create the final unit CPG, the recurrent cyclic inhibition network. Three identically tuned ENs were connected via identical inhibitory synapses. Results are seen in Fig. 8, below. In Panel A, intrinsically active spiking neurons show the characteristic 3-phase network activity of such a circuit. While RC1 fires, it inhibits RC2, which stays silent until it experiences post-inhibitory rebound due to the fact that RC1 has stopped firing. While RC2 fires, it inhibits RC3. Meanwhile RC1 is rising, but does not fire again before RC3, now released from inhibition by RC2, fires and inhibits it. When RC3 finishes its burst, it causes RC 1 to fire a post inhibitory rebound induced volley and the cycle repeats. Panels B and C show the same circuit, for which the ENs have been tuned to be intrinsically silent. The network is started by a brief current injection to either RC1 alone (B), or to all three 11

12 neurons (C). Notice the odd behavior of the neurons in C as all three neurons are receiving the same input, and also inhibiting each other at the same time. The network begins to act once the current injection ceases and some asymmetry kicks the network into action. Electronic Neuron Parameters G(ohms) 50 N(ohms) 0 M(ohms) (v) I (v) A 0.833; B,C: 0.4 e (v) h (v) -2.7 Figure 8: A simple recurrent cyclic inhibition CPG made with ENs. In A, three intrinsically active neurons are connected via identical inhibitory synapses. B shows intrinsically silent neurons that begin to show network activity when RCl is activated by a brief current pulse. C shows the action of the network triggered by a current pulse delivered to all RCs at once. DISCRETE-TIME MAP-BASED NEURONS Map-based neurons (MBNs), like the electronic neurons above, were developed at the INLS and have been implemented in Labview, the same program used to collect analog data from the ENs, above. MBNs are mathematically modeled neurons created by a system of two difference equations (see Fig. 9). As they are implemented in Labview, they are discrete-time (digital) neurons, and therefore are temporally constrained by the limitations of the computer processing hardware and the programming used to create them. Despite this, because of the relatively few calculations needed per iteration, map neurons can be run in real-time. This is imperative if one plans on controlling robotics or prosthetics with artificial neurons, but it is very hard to create artificial neurons that run in real time and also can capture the dynamics and biophysical characteristics of real neurons. This is because generally these characteristics are borne out of interactions between many currents in the biological cells that are inherently computationally intensive to model. It is rare that a phenomenological model 12

13 such as is exemplified by these MBNs is able to display the wide range of network activities demonstrated below. RULKOV TWO-DIMENSIONAL MAP EQUATIONS In the equations below, Xn controls the fast dynamic of a spiking neuron (namely, the action potential) and represents the transmembrane potential, while Yn underlies the slow wave phenomena that gives rise to bursting (Rulkov 2002). The parameters controllable in the Labview programming interface are a, which mainly modifies the fast dynamics of the neuron, and cr, which underlies the slower dynamics and controls the bursting behavior of the neurons. U is a constant equal to 0.01, and the variables lx,y are the current inputs from modeled chemical synapses, described below. Rulkov Map-based Neuron Equations (8) x<o 0 < X < a+y +J.!Ix x>a+y (9) (10) Figure 9: Rulkov Two-dimensional map-based neuron equations, adapted from Rulkov u controls the fast, and u the slow dynamics of the MBN. The symbol mu is a constant, Ix,y are currents due to synaptic connections (see Fig. 12). See Rulkov 2002 for more information. Xn travels around a map similar to the one shown in Fig. 10. One can observe that whenever Xn is positive, the next iteration, Xn+l, will be equal to 2, the maximum value for this particular MBN, and that Xn+l will then necessarily be equal to -1. This particular orbit gives rise to the spiking behavior. Varying x based on y changes the properties of the map such that x will spend time alternately in the spiking region of the map, given by Pk, and otherwise it will remain in the nonspiking region. This gives rise to bursting behavior. Therefore, by changing the parameters a and cr, the two dimensional neuron can be made to display properties of tonic spiking, and spiking and bursting behaviors. A parameter map and an accompanying figure depicting different MBN types can be seen in Fig

14 2 Figure 10: The map after which the map-based neurons + 0 r:::. -1 are named. The thick black line is given by Eqns. 8, 9 and 10. From Rulkov a b 5 (Jth Bursts of Spikes.c::$ 4 Silen e Spikes Chaotic Oscillations (J' Figure 11: A parameter map for a and s shows three different regimes of MBN behavior. On the right are examples of MBN waveforms with the parameters determined by the dots shown on the parameter map. Adapted from Rulkov, SYNAPSE EQUATIONS Multiple MBNs may be connected in circuits via modeled synapses (MBNS) to affect the output of other neurons. The MBNS equations are shown in Fig. 12, below, and are a system of equations that gives values for lx and ly based on Zn. Zn is given by its previous value, Zn-1, time a relaxation coefficient, r, between 0 and 1. r determines the characteristic time constant of the synapse. k is a logical variable that gates the second term of Eqn. 13; it is equal to 0 if there is 14

15 no presynaptic action potential detected in the previous iteration, and it is equal to 1 if there was a presynaptic action potential detected. Therefore, the value of the second term is added to z only if there was an action potential detected. g regulates the synaptic strength, x is the postsynaptic cell's value for Eqn. 8, and Xrev is the reversal potential for the synapse, which again dictates whether the synapse will be excitatory or inhibitory. Once Ix and ly are determined, they are fed into Eqns. 9 and 10 of the postsynaptic cell, inhibiting or exciting it as the case may be. Rulkov Chemical Synapse Equations = (0.3)*zn (11) = Zn (12) where: zn = r * zn-l- k*g*(x- ev) (13) Figure 12: Rulkov chemical synapse equations. lx,y are currents due to synaptic connections (see Fig. 9), r is the relaxation rate (max=l), k is a logical variable that equals one when there is a spike and 0 when there is no spike, g is the synaptic strength and Xrev is the reversal potential of the synapse. BUILDING BASIC CPGs WITH MAP-BASED NEURONS The MBNs and MBNSs were then used as described above to model the three basic CPG types. This was done using the Labview programming interface, which allows for rapid network construction and prototyping of neuronal phenotype through modularity and an intuitive graphical interface. This approach is much quicker than building networks with ENs as the ENs have to be physically wired together and tuned by hand using a screwdriver and multimeter. Wiring a network in Labview is as easy as seen in Fig. 13, and the inherent modularity lets one use pre-made networks over and over again. Tuning MBNs and synapses, and visualizing the results of simulations is easy as well. The front panel interface, seen in Fig. 14, allows for visualization of the network's output, as well as rapid adjustment of the parameters of any part of the network. 15

16 At the very lowest level lies the equations that describe the map-based neurons and chemical synapses. To the the right io " grophic:>l repreoenbtion of a 2-D l.\o1bn. The equations sit inside 'formula nodes' which in tum sit inside a 'while loop'. "''" ilyluts. outputs Any program in labview may be saved and called to run within another program, in which it will be represented by a single icon. The figure on the left shows one half of a reciprocal half center CPG, with a l.\o1bn, and synapses that will deal with inputs from command cells and the other half of the CPG c.., synapses (;> (;> - 6> :;::: 'ifll!lse' stateromts Programs may be nested further; the figure on the right(c) represents an entire reciprocal half center CPG, containing two copies of the program represented above (B), and additional support programming. Once the basic CPGs are created in Labview, they can be used to create more complex networks. D shows the 1rue modularity of the labview programming interface, three half centers, each containing A, B, and C as described above are connected via inhibitory synapses, creating the alternating tripod locomotor pattern. Figure 13: The Labview 'block diagram' progranuning envirorunent. Nesting programs allows for modularity that enables quick assembly and rearrangement of complex circuitry. 16

17 :G;. (. 0' (? (.' (,{. parameters may be adjusted with knobs, switches, sliders or number boxes Figure 14: This is the Labview 'front panel' where the programming seen in Fig. 1 is controlled and the results output. On the left are the parameter controls and on the right is the graphical output of the network. This interface is the basis for rapid prototyping of cell type and network connectivity characteristics. ENDOGENOUS BURSTER Using the Labview interface, MBNS, and MBNSs as described above, a basic endogenous burster CPG was created. Fig. 15A shows an endogenously bursting MBN, EB, and an intrinsically silent follower MBN, F, that are not connected via any coupling. When an excitatory synapse is present, as in B, the neurons fire together. If an inhibitory synapse is present instead, EB and F will fire out of phase, F' s spiking caused by post inhibitory rebound. Map-based Neuron Parameters Figure 15: a simple endogenous EB 5.2 burster CPG a F: 3.5 constructed with MBNs and EB 0 a MBNSs. Panel A F: 0.12 shows endogenous Synapse Parameters burster EB and follower F when AO no synapse is strength B: 0.1 present. B shows c 0.01 the addition of an relax 0.98 excitatory, and C the addition of an reversal B: 2.2 inhibitory synapse. potential c -2.2 MBN andmsns parameters are on iterations 2000 the right. 17

18 RECJPROCAL HALF CENTER MBNs were then used to create a reciprocal half center CPG. Two identical, intrinsically spiking MBNs, HC1, and HC2, were connected via identical inhibitory synapses. Fig. 16A shows results of the first 4000 iterations in the absence of command stimulation. HC 1 and HC2 fire antagonistic bursts, and continue doing so indefinitely. When HC1 and HC2 are made intrinsically silent by reducing the value of a, and an excitatory synapse is introduced between a command neuron (cmd) and HC1, the CPG was triggered to activate its oscillatory rhythm as seen in B. Cmd A Map-based Neuron Parameters a 3.5 (J A: 0.15 B: 0.12 Synapse Parameters strength A,B(HC): 0.1 B(Cmd): relax 0.98 reversal A,B(HC): -2.2 potential B(Cmd): 2.2 iterations 4000 Figure 16: two identical MBNs make up a reciprocal half center CPG. In A, MBNs are intrinsically active spiking neurons connected via inhibitory synapses. In B, MBNs are intrinsically silent, but begin to oscillate after a brief input from an excitatory synapse to HCl from Cmd. RECURRENT CYCLIC INHIBITION The final basic CPG, the recurrent cyclic inhibitory CPG, was also constructed with MBNs. Three identical intrinsically silent spiking neurons, RC1, RC2, and RC3, were connected via inhibitory synapses. Fig. 17 A shows the activity of this network after an excitatory stimulus is introduced to RC1 through an excitatory synapse from command neuron, Cmd. Notice the three phase network behavior, as described above, and that the burst of spikes in any given neuron is due to its post inhibitory rebounding. Panel B shows the same network, but for which all three neurons were connected to Cmd via an excitatory synapse. Notice the synchronous firing during excitation and the subsequent initiation 18

19 of the three phase pattern once some asynchrony has been introduced. Notice also that the burst duration and interburst interval of these neurons are smaller in this network than in A. Cmd RCl A Map-based Neuron Parameters a 3.4 RC RC3 Synapse Parameters Cmd strength A,B(RC): B(Cmd): RCl RC2 relax 0.98 reversal potential A,B(RC): -2.2 B(Cmd): 2.2 iterations 4000 RC3 Figure 17: A recurrent Cyclic Inhibitory network built with MBNs and MBNSs. A and B show identical intrinsically silent neurons RCl, RC2, and RC3 connected via identical inhibitory synapses. In A, network activity is initiated by a burst of spikes from Cmd which synapses onto RCl with an excitatory synapse. In B, all three RCs receive excitation from Cmd. COMMAND NEURON, COORDINATING NEURON, CPG ARCHITECTURE An important aspect of communication between small circuits is in the area of command input to CPGs and coordination between CPGs via coordinating neurons. This becomes very important in interlimb phase control. In order to assure that CPGs in different limbs are acting in concert within the same motor program, a command neuron-coordinating neuron-cpg architecture has been suggested (Stein 1978). In this design, a command neuron comes from higher brain centers and acts upon separate CPGs, each of which controls a limb, or some subset of the overall motor program. Using this architecture, motor programs can be completely contained outside of the decision making centers of the brain, and only activated once a decision has been made by these regions. In most vertebrates, this architecture usually has the brain receive sensory input from cranial and peripheral nerves in the spinal cord, process this information, and send a motor command or commands to the spinal cord, where the CPGs that make up motor programs lie. In invertebrates, such as the lobster, the architecture is similar, but instead of CPGs residing in spinal cord (as invertebrates have none), they lie in various ganglia. In addition to receiving input concerning when to fire and when to be silent, CPGs must also be able to obtain information about the operation of other CPGs, which may be using the same 19

20 muscle synergies, and they must be able to modify their phases relative to the activity of all the other limb CPGs. The carrier of this information has been dubbed the "coordinating neuron" (Stein 1978) and neurons that carry this information have been found in many biological networks. For example, a coordinating neuron may receive synaptic input from the endogenous pacemaker of a "governing" CPG and send excitatory input to the pacemaker of a "governed" CPG. The bursting rhythm of the governing CPG would then set, or entrain, the timing of the governed CPG. Conversely, if the reversal potential of the synapse is changed to make it inhibitory, the two CPGs would then fire out of phase due to the effects of inhibitory coupling. There are many other examples of coordinating information, such as the phase lag introduced in metachronal pattern generation as is seen in the hemicord of a swimming vertebrate. Exteroceptive information introduced to coordinating neurons may be responsible for adaptive reflexes such as load compensation or the stumble reflex. COMPLEX NETWORKS BUILT WITH MBN S: COMMAND AND COORDINATION It was therefore important to show that the MBNs and ENs could both receive command and coordinating information from command cells and from other CPGs. Figures 7, 8, 16, and 17 clearly show CPGs whose behavior is initiated by input from command sources. In order to construct networks that displayed properties of coordination between CPGs, MBN s were used to construct the various networks seen in Figure 18. Figure 18A shows a network in which a bursting neuron (EB) and a follower (F) are both being driven by command input from Cmd. EB inhibits F, so they fire out of phase with EB setting the bursting rhythm. In B, if a coordinating cell (P) perturbs EB with a excitatory connection, it resets the timing of EBs bursts (compare to the timing in A) and therefore also resets F's timing. When P stops firing, EB and F return to the timing seen in A. In Figure 18C, a metachronal rhythm-generating network is depicted. Cells a1-4 are all excited by a command cell, Cmd. Cell a1 has slightly stronger input from Cmd, than does a2, which has slightly stronger input than a3, etc. In addition, coordinating delay neurons, c1.3 are connected between each of the 'a's. Since the command input to a2 is less than to a1, a1 fires first, excites c1, and this causes c1 to inhibit a2. a2 does not fire until c1 stops firing, and in this way, a phase lag between each level of the 'a's can be introduced. This is the basis for the lag in the successive firing of segments in a hemicord of a swimming chordate. One can imagine each 'a' being reciprocally connected to the 'a' of the same segment on the opposite side of the spinal chord via an inhibitory connection, and so produce an oscillatory pattern across the spinal cord that travels caudally via the network mechanism shown in Figure 18C. In Figure 18D, the line between coordinating neuron and CPG begins to blur. This is the alternating tripod locomotor network that serves as the basis for insect walking. Any two adjacent cells are connected via reciprocal inhibitory synapses, and so taking any two adjacent cells into consideration, one will observe a reciprocal half center CPG. If one imagines that the half center pairs are: a and b, c and d, and e and f, then one will see the connections between a and c, or those between d and f for example, as coordinating neurons, entraining adjacent CPGs to a common rhythm. However, it may also be possible to see this network as a series of 20

21 interconnected reciprocal half center CPGs, not considering coordinating neurons at all. In any case, a two phase pattern emerges from this network in which a, d, and e all fire synchronously and exactly out of phase with b, c, and f, which fire synchronously together. Figure 18: Five more complex networks built with MBNs. Briefly, A and B demonstrate the principle of coordination of CPGs, C shows the command-coordinating-cpg network in a metachronal rhythm such as might be observed in a swimming motor progt am, D shows the insect alternating tripod locomotor gait and E shows the lobster walking motor program. See text for details. Figure 18E shows the locomotor walking program of the lobster. Endogenous burster 'ele' (elevator), inhibits both 'dep' (depressor), and 'st' (stance). St then inhibits 'sw' (swing). This simple CPG sits at the base of each leg of the lobster and receives command input from higher brain centers as well as coordinating input from the other copies of itself at each leg, ensuring that the legs are all in the correct phase relative to one another. The above networks demonstrating the properties of command and coordination of CPGs were all created with MBN s in the Lab view programming interface. This interface allows one to create modular networks and reset and rearrange parameters and connections very rapidly when compared with the ENs (see Figure 13). It would be eventually desirable to create sensory integration and command networks with MBN s, while ENs would implement CPGs and the coordination between them. The rationale for this approach is outlined below. 21

22 HYBRID NETWORKS: INTERFACING ANALOG ELECTRONIC NEURONS WITH DIGITAL MAP-BASED NEURONS Once it was shown that it was possible and relatively straightforward to build basic CPG circuits with both ENs and MBN s, a hybrid architecture, in which MBN s could affect ENs and ENs could affect MBN s was studied. This hybrid architecture would take advantage of both the biologically more realistic ENs, and the computationally less intensive and rapidly prototyping MBN s. In order to accomplish this, it was necessary to acquire analog data from the ENs, digitize it, and allow circuits to be built in Lab view that incorporated data from both ENs and MBN s. To affect ENs, it was necessary to also convert some digital data into analog data and feed this into ECSs within an EN network. Results are shown below, but first, the motivation for the hybrid architecture will be discussed. WHY A HYBRID ARCIDTECTURE? The most important reason for the hybrid architecture is that it will allow the creation of circuits that include both the speed and computational efficacy of the MBN s with the biological realism of the ENs. This is important in the creation of a control system for biomimetic robotics or neural prostheses. The MBN s are able to handle the computational load of sensory integration and processing and to send simple, coherent controls to the CPG and coordinating networks constructed of ENs, which will actually drive the biologically realistic locomotor pattern generation networks and actuators. UCSD Hindmarsh-Rose electronic neurons, as described above, are based on four differential equations. The differential equations were created by analyzing data from thousands of biological neurons and studying the degrees of freedom granted by the observed behaviors. It was found that there were seven degrees of freedom in these cells, and these are represented by the seven adjustable parameters present in the ENs. This implies that the ENs should be able to reproduce virtually any characteristic of the neurons' behavior used in their creation. In contrast, the MBN s are modeled by two difference equations, with only two adjustable variables to account for their behavior. This implies that they do not have the complexity required to reproduce all of the possible waveforms that could be observed in biological cells. They do spike and burst, but these properties are confined to the map created by the a and a parameters as seen in Fig. 10. It is important to note that both the ENs and MBN s are phenomenological models, meaning that they reproduce the waveforms seen in biological neurons, but they do not do so by mimicking the mechanisms of real neurons. In other words, these are not models based on conductance models of membrane voltage and action potential formation, but rather are equations that roughly approximate fast and slow dynamics (MBNs), or membrane voltage, fast, slow and calcium dynamics (ENs). Since the ENs have seven parameters roughly based on different properties of biological neurons, and MBN s only have two, it is clear that the combinatorics of the ENs allow for many 22

23 more parameter combinations, and therefore a greater variety of neuronal phenotype than do the MBNs. It would seem relatively straightforward, then, to use ENs in all simulations of neural networks, since ENs can provide a greater variety of cell types than will MBN s. There are some drawbacks of ENs, however. First, they are much more computationally expensive than MBN s, requiring more time, power, and processing power to operate. Secondly, ENs are much more difficult to tune and to connect into networks than are MBNs. To tune an EN, one must use a multimeter and constantly monitor the resistance or voltage at different parts of the circuit while adjusting a variable resistor with a screwdriver. This introduces variability and a very large time burden into the tuning of ENs. Network construction also requires that networks be rebuilt every time they are needed, and it requires one circuit board for each EN and one for each ECS, with one to two wires running between each board for each synaptic contact. In contrast, MBN s, which only require the calculation of two difference equations, are much more computationally efficient. In addition, the Labview interface renders tuning and connecting neurons together a very simple task. Further, any network, once created, may be saved and called from another, more complicated network without needing to rebuild and retune it. It is this modular nature of programming in Labview, as seen in Fig. 13, that is the greatest advantage to building networks in Lab view. Therefore, it would be beneficial to retain ENs where it is necessary to have biologically realistic neurons, and to use MBN s in all other instances in order to reduce the computational and labor intensive nature of the network building requirements. The places it is most important to have biological realism are in the CPG networks themselves, as it is imperative that the cells in these networks are able to mimic biophysical properties seen in biological CPGs. It has been shown above that both the ENs and MBN s can create basic CPGs, and so one might wonder why it is necessary to use ENs at all. The answer lies in more complex types of cellular membrane voltage activity. There are certain biophysical properties of neurons, not studied here, that can modify CPG activity in terms of response to input, as well as in initiation and termination properties. There is preliminary evidence that the ENs will be better at modeling these types of activity than will be the MBNs. With their seven degrees of freedom over the MBNs' two, it is not entirely surprising that the ENs would be better able to mimic these properties. Command and coordinating networks do not rely so heavily on exotic biophysical properties but more on proper connectivity and timing. Since the MBNs are perfectly sufficient in providing these types of connections, as evidenced in Fig. 18, above, it seems logical to use MBNs in this role. In this vein, hybrid networks were created, exploring the various roles of ENs and MBN s in creating functional biologically-inspired neural networks. MAP-BASED NEURONS SYNAPSING ONTO ELECTRONIC NEURONS In order to create a functional synapse between the MBN s and the ENs, it was necessary to create an analog output from the waveform of the presynaptic MBN. This was accomplished using Labview' s built in digital acquisition software, which comes supplied with virtual instruments that can be added to the block diagram of a program, and accomplishes both analogdigital acquisition and digital-analog output functions. These were used to change one of the outputs on the terminal block (mentioned above) to match the voltage of the desired MBN in 23

24 each cycle of the simulated program. This output on the terminal block was in turn connected to the presynaptic input on an ECS, which was in turn connected to the desired postsynaptic EN. Analog-to-digital and digital-to-analog signal processing was accomplished using the digital acquisition board mentioned above. The sampling and display rate was 400-SOOHz for Figs. 19&21, and 1000Hz for Figs. 20&22. Figure 19A shows an EN in the ab sence of any synaptic input. When An MBN is connected to it via an ECS, as described above, and the reversal potential of the ECS is set such that the synapse is excitatory, the MBN will excite and entrain the EN to the firing rhythm of the MBN. If instead the synapse is made to be inhibitory, the MBN will inhibit the EN and again entrain its rhythm. This shows the proof-of-concept of interacting the MBNs with the ENs, and opens the possibility for using MBNs as command and/or coordinating cells for EN networks. Iterations: Figure 19: MBNs used to affect ENs. Panel A shows an endogenously bursting EN in the absence of any synaptic input. B shows an MBN that is presynaptic to the EN via an excitatory synapse. The MBN changes the burst interval and burst duration of the EN. In C, the neurons are connected via an inhibitory synapse, ENs activity is again altered, it's spiking and bursting a combination of its intrinsic activity and post inhibitory rebound. Along these lines, a recurrent cyclic inhibitory network comprised of ENs was created, with the members of the network all being intrinsically silent (see figure 20, below). An MBN was connected to RCl via an ECS tuned to be excitatory and when it displayed a short burst, caused the RCI network to be active. 24

25 Cmd RCl RC2 RC3 ENs set same as revious RCI exam le lvlbn-7en Synapse Parameters Vslope (Mohms) 1.0 Strength (Kohms) 620 Vthresh (v) 2.55 Vrev (v) 5.7 Iterations 8000 Figure 20: Cmd, an MBN, plays the role of command neuron to this recurrent cyclic inhibitory CPG made up of ENs. Cmd synapses onto RCl and excites it to cause the network to begin its activity. The previous two figures show that it is possible to use an MBN to entrain an EN and also to trigger a network. In other words, it is possible to use MBNs as both command and coordinating neurons in a hybrid network made of MBN s and ENs. The following section describes experiments to communicate in the other direction; from ENs to MBNs. ELECTRONIC NEURONS SYNAPSING ONTO MAP-BASED NEURONS In order to create an input to one or multiple MBNs from an EN, it was necessary first to accomplish some analog to digital conversion, in order to have the data accessible to the digitally-computed MBNs. This was accomplished by using the Labview terminal block and digital acquisition board mentioned above to read a single value for each of the MBN program's iterations. The values for two previous iterations were also stored in the program and used to determine when the EN was spiking. This was determined to be when the derivative of the EN' s voltage was sufficiently positive on one iteration, and followed by a negative value on the next iteration (corresponding to a sharp maximum, as seen during an action potential). The presence of an action potential was fed into an MBNS as the variable 'k' in Eqn. 13. Therefore, the value of k was made to be 0 during program cycles when the EN was determined not to have spiked, and 1 during program cycles when the EN was determined to have spiked. Figure 21 shows an MBN in the absence of any synaptic input. It is intrinsically silent. When an excitatory synapse is introduced with a bursting EN as the presynaptic source as seen in Panel B, the MBN fires as it receives synaptic excitation from the EN. The vertical hatch marks beneath the EN waveform represent action potentials detected by the MBN program. If the synapse between the EN and MBN is instead made to be inhibitory, as seen in Panel C, the MBN clearly shows post inhibitory rebound and fires a burst of action potentials in response to a release from inhibition. 25

26 MBN A ---- Map-based Neuron Parameters MBN EN MBN Iterations: 1000 Figure 21: Here an EN synapses onto an MBN. Panel A shows the intrinsically silent MBN to which there is no input. In B, an excitatory MBNS has been introduced between the EN and the MBN, causing the MBN to fire in phase with the EN. C shows the same network where the synapse from EN to MBN is inhibitory instead of excitatory. The vertical bars beneath the EN show where the program determined EN was spiking. TWO-WAY SYNAPTIC ACTIVITY A RECURRENT CYCLIC INHIBITION CPG USING BOlli ELECTRONIC NEURONS AND MAP-BASED NEURONS Once it was proved that one-way synaptic input could be conferred both from ENs to MBNs or from MBNs to ENs, a network that required two-way synaptic connectivity was attempted. In this experiment, on of the RC ENs was removed from a recurrent cyclic inhibitory network and replaced with an MBN. Figure 22 shows the results of this experiment, with MBNs and ENs clearly inhibiting each other and giving rise to the characteristic three-phase recurrent cyclic inhibitory network pattern. Whereas in other experiments, the burst durations of RC neurons were approximately equal, it can bee seen that this is not the case here. One might expect an asymmetry in the burst durations between EN and MBN members of this network, as the two neuron types are not equivalent, and it isn't entirely feasible to tune them to respond in exactly equivalent manners. The cause of the asymmetry between the burst durations of ENl and EN2 may not be as directly apparent, but the cause is similar to the difference between ENs and MBNs. This is due to different amounts of inhibition reaching ENl from MBN than are reaching EN2 from EN l. Since the burst duration of MBN is shorter than that of EN l, ENl receives inhibition for a shorter duration than does EN2. The mechanics of the post inhibitory rebound are such that this causes EN2 to have a longer burst of action potentials than EN 1. 26

27 Figure 22: A recurrent cyclic inhibition network made up of one MBN and two ENs. The burst durations are not equal because there is asymmetry inherent in using two different types of modeled neurons. The threephase pattern is clearly visible however and it is clear that the network activity is due to rebound from inhibition. DISCUSSION Three basic CPG types were created with each of two different circuit media, MB N s and ENs, and were both able to reproduce patterns seen normally in biological systems. fu addition, the command and coordinating properties of some more complex networks were explored. It is clear that both discrete time map-based neurons and UCSD Hindmarsh-Rose electronic neurons can support the processes necessary for central pattern generation, and the command modulation and coordination of these CPGs. Discrete time map-based neurons and the Labview programming interface provide a rapid prototyping environment for the development of neurotechnology, and the electronic neurons provide a more biologically realistic model that can account for more of the naturally occurring neuronal waveforms. Discrete time map-based neurons can be interfaced directly to electronic neuronal networks to form hybrid electronic nervous systems that take advantage of the benefits of each of these modeled neuron types. The proper use of ENs and MBNs in a hybrid circuit can maximize the benefits and minimize the drawbacks of each and create circuits that are adequate for the control of biomimetic robots and neuroprostheses. We are still investigating the initiation and termination of small network activity, which was not covered here. fu biological networks, cellular and synaptic properties of fatigue and antifacilitation lead to the decrease in activity of a given network. Conversely, properties of synaptic facilitation cause synapses to grow stronger over time, and give rise to things like learning, memory, and other adaptive features. Additional command mechanisms, such as neuromodulation, must also be modeled. In ENs, this could be accomplished by creating a circuit or some lines of code, which models the rise and fall times of the current input to a cell caused by the concentration of different neurotransmitters, and simply adding this to an ENs' 'I' 27