Fundamental mechanisms of visual motion detection: models, cells and functions

Transcription

1 Progress in Neurobiology 68 (2003) Fundamental mechanisms of visual motion detection: models, cells and functions C.W.G. Clifford a,, M.R. Ibbotson b,1 a Colour, Form and Motion Laboratory, Visual Perception Unit, School of Psychology, The University of Sydney, Sydney 2006, NSW, Australia b Centre for Visual Sciences, Research School of Biological Sciences, Australian National University, Canberra 2601, ACT, Australia Received 8 May 2002; accepted 12 November 2002 Abstract Taking a comparative approach, data from a range of visual species are discussed in the context of ideas about mechanisms of motion detection. The cellular basis of motion detection in the vertebrate retina, sub-cortical structures and visual cortex is reviewed alongside that of the insect optic lobes. Special care is taken to relate concepts from theoretical models to the neural circuitry in biological systems. Motion detection involves spatiotemporal pre-filters, temporal delay filters and non-linear interactions. A number of different types of non-linear mechanism such as facilitation, inhibition and division have been proposed to underlie direction selectivity. The resulting direction-selective mechanisms can be combined to produce speed-tuned motion detectors. Motion detection is a dynamic process with adaptation as a fundamental property. The behavior of adaptive mechanisms in motion detection is discussed, focusing on the informational basis of motion adaptation, its phenomenology in human vision, and its cellular basis. The question of whether motion adaptation serves a function or is simply the result of neural fatigue is critically addressed. Crown Copyright 2003 Published by Elsevier Science Ltd. All rights reserved. Contents 1. Introduction General motion detector mechanisms Fundamentals of motion detection Pre-filtering On- and Off-channels Temporal characteristics of pre-filters Temporal delay filtering Non-linear interactions Facilitation Inhibition Speed-tuned motion detectors Evidence for the cellular mechanisms of motion detection Retinal motion detectors in vertebrates Sub-cortical motion processing Cortical motion processing Motion detectors in insect optic lobes Abbreviations: AOS, accessory optic system; APB, 2-amino-4-phosphonobutyric acid; DAE, direction aftereffect; DS, direction selective; DTN, dorsal terminal nucleus; fmri, functional magnetic resonance imaging; GABA, -aminobutyric acid; ISI, inter-stimulus interval; LGN, lateral geniculate nucleus; LTN, lateral terminal nucleus; MAE, motion aftereffect; MST, medial superior temporal area; MT, middle temporal area (V5); MTN, medial terminal nucleus; NOT, nucleus of the optic tract; PMLS, posteromedial lateral supersylvian area; RGC, retinal ganglion cell; STOLF, space time oriented linear filter; TFRF, temporal filter response function; V1, primary visual cortex (area 17); V5, middle temporal area (MT); WIM, weighted intersection model Corresponding author. Tel.: ; fax: addresses: colinc@psych.usyd.edu.au (C.W.G. Clifford), ibbotson@rsbs.anu.edu.au (M.R. Ibbotson). 1 Tel.: ; fax: /03/$ see front matter Crown Copyright 2003 Published by Elsevier Science Ltd. All rights reserved. doi: /s (02)

2 410 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Adaptive mechanisms in motion detection Perceptual consequences of motion adaptation Function or fatigue? Informational basis of motion adaptation Dynamics of motion adaptation Directionality of motion adaptation Distinguishing motion adaptation from contrast adaptation Concluding remarks References Introduction While numerous visual animals lack color or binocular vision, the ability to see motion is ubiquitous and, next to the detection of light and dark, may be the oldest and most basic of visual capabilities (Nakayama, 1985). Consequently, visual motion processing is of fundamental interest to systems neuroscience and has been the subject of intense research. The present paper reviews recent advances in our understanding of motion detection in biological systems in the context of the large body of work that has gone before. In particular, we emphasize the contribution made by comparative studies of motion detection to the broader understanding of the topic. The modern theoretical framework for motion detection was developed from behavioral experiments on the Chlorophanus beetle (Hassenstein and Reichardt, 1956; Reichardt, 1961). The relevance of this early work to subsequent studies of motion detection, including primate cortical physiology and human psychophysics, indicates the importance of a broad biological approach. A key feature of motion processing to emerge at both the cellular and systems levels is its dynamic nature and adaptive plasticity. Consequently, motion adaptation will be a major focus of this review. Perhaps the next great challenge in understanding motion detection is to reconcile the wide range of theoretical approaches with the cellular basis. Given the important advances that have already been made in this direction, we review progress on both of these levels. The moving world is projected onto the retina in the form of a spatiotemporal pattern of light intensity. From this dynamic signal, recovery of the direction of image motion is the first stage of extracting behaviorally relevant information. Section 2 of this review will cover motion processing from initial non-directional filtering strategies up to the point where a directional signal is produced. Section 3 then deals with the evidence for cellular mechanisms that might perform these tasks in a range of brain areas and species. Specifically, we look at motion processing in the vertebrate and insect visual systems. Section 4 deals with the important role performed by adaptive mechanisms in motion processing. 2. General motion detector mechanisms 2.1. Fundamentals of motion detection Exner (1894) was the first person to discuss the requirements necessary for generating a motion signal from neural circuitry. He presented a drawing of a neural network that can be regarded as the first attempt at a motion detector model (Fig. 1A). However, it was another German scientist, Reichardt (1961), who promoted the first computationally based model of motion detection (Hassenstein and Reichardt, 1956), a model that has subsequently been given his name (Fig. 1B). Although motion detector models vary in their detailed structure, the Reichardt detector is useful in setting out the basic framework necessary for motion detection (e.g. Borst and Egelhaaf, 1989). Detecting the direction of motion requires that the image be sampled at more than one position or spatial phase, that these samples be processed asymmetrically in time, and that they be combined in a non-linear fashion (Poggio and Reichardt, 1973; Borst and Egelhaaf, 1989). This is a serial process that involves computation at multiple synaptic levels. These stages will be covered in three sub-sections: pre-filtering, delay filtering and non-linear interactions Pre-filtering While Reichardt s (1961) original model included spatial and temporal pre-filters (Fig. 1B), many subsequent models of motion detection have neglected the importance of pre-filtering (although see van Santen and Sperling, 1984, 1985; Ibbotson and Clifford, 2001a,b). Pre-filters are important because their properties affect the tuning characteristics of the motion detectors they feed. What do we know about the pre-filters of biological motion detectors? On- and Off-channels It is well established that vertebrate photoreceptors are hyperpolarized by light and their outputs are fed into bipolar cells. Sign conserving synapses feed Off-bipolar cells while sign inverting synapses feed On-bipolar cells (Werblin and Dowling, 1969) such that On-cells are excited only

3 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Fig. 1. Two proposed delay and compare schemes for motion detection. (A) Schematic of neural center for motion perception proposed by Exner (1894). Retinal fibers feed into points a f and similar points. Signals from these points are summed at sites S, E, J f and J t. The time taken for a signal from a given point to reach a site of summation is proportional to the distance that signal must travel. It is this delay between signals from different retinal locations that introduces directionality into the scheme. (B) Schematic of the mathematical model proposed by Reichardt (1961) to describe the optomotor response to motion stimuli in the Chlorophanus beetle. Ommatidia A and B are separated by an angular distance, s. The temporal responses, L A and L B, from these receptors are linearly transformed by the units D, F and H and linked together in the multiplier units, M A and M B. The outputs of the multiplier units are passed through low-pass temporal filters, S A and S B, and then subtracted from each other. The output of the subtraction stage controls the motor response of the beetle. by brightness increments (On stimuli) while Off-cells are excited only by brightness decrements (Off stimuli). Just as the On- and Off-cells are excited by opposite brightness polarities, they are also inhibited in a polarity dependent fashion, so On-cells are inhibited by Off stimulation and Off-cells by On stimulation (e.g. Enroth-Cugell and Robson, 1966; Hochstein and Shapley, 1976). Since Onand Off-cells are inhibited by brightness decrements and increments, respectively, why are both On- and Off-channels necessary when a single channel might suffice to carry information about both brightness polarities? Schiller et al. (1986) suggest that coding information about both increments and decrements through opposing excitatory processes is much more efficient than using excitation and inhibition within a single channel. They reason that the low spontaneous activity of retinal ganglion cells (RGCs) means that inhibition below this level cannot represent much information, while a higher spontaneous rate would have a high metabolic cost (Laughlin et al., 1998). The coding of brightness through On- and Off-channels might thus be the most efficient way to satisfy both informational and metabolic constraints. How do signals from the On- and Off-channels interact in the generation of direction-selective responses? A classic example of the way that pre-filtering affects the outputs of directional neurons comes from rabbit retina, where two distinct types of direction selective (DS) retinal ganglion cells have been identified: On-DS cells and On Off-DS cells (Barlow et al., 1964; Barlow and Levick, 1965). Both cell types generate direction-selective responses. On-DS cells respond to the movement of bright bars while On Off-cells respond to the movement of both bright and dark bars. Systems in other species show qualitatively different interactions between On- and Off-channels. For example, DS cells in the pretectal nucleus of the optic tract (NOT) of the marsupial wallaby (Ibbotson and Clifford, 2001a), the primary visual cortex of the cat (Emerson et al., 1987, 1992) and macaque middle temporal area (Livingstone et al., 2001) show interactions between On- and Off-signals that utilize the sign of the incoming signals. In the insect visual system, the retinal image is not segregated into On- and Off-channels and wide-field direction-selective neurons in the fly optic lobe receive input from motion detectors whose pre-filters maintain brightness polarity (Egelhaaf and Borst,

4 412 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Fig. 2. Stimulus sequences and space time plots of phi and reverse-phi motion. (Top left) Frames of an image sequence taken at times T1 T5 show a white bar moving across a gray background at constant speed. Seen in succession, the image sequence gives rise to the perception of phi apparent motion (Wertheimer, 1912), the bar appearing to move across the background. The sequence of frames may be placed together to form an image volume (top right), with time as the third dimension. (Bottom left) A slice through this space time volume illustrates the fact that motion is equivalent to space time orientation. (Middle row) A sequence of images in which the contrast polarity of the bar reverses between black and white each time it moves. This image sequence gives rise to the percept of reverse-phi motion in the direction opposite to the displacement of the bar (Anstis, 1970; Anstis and Rogers, 1975). (Bottom right) Space time plot of the reverse-phi motion stimulus. 1992). DS cells with inputs whose sign preserves brightness polarity show characteristic responses to apparent motion stimuli consisting of increases or decreases in brightness. Sequences of brightness steps of like polarity (either increments or decrements) elicit positive motion-dependent response components to motion in the preferred direction and negative responses to motion in the anti-preferred direction. For sequences of opposite polarities, these directional properties are reversed (Fig. 2). These response properties are reminiscent of the reverse-phi phenomenon in human vision (Anstis, 1970; Anstis and Rogers, 1975). For a wide range of spatial and temporal displacements, humans perceive sequential brightness changes at neighboring positions in the visual field as motion in the direction of the second brightness change ( phi motion : Exner, 1875). When the sequential brightness changes are of opposite contrast polarity, motion in the reverse direction is perceived ( reverse phi motion : Anstis, 1970; Anstis and Rogers, 1975). How are the On Off interactions evident from psychophysics implemented in the primate visual system? Using an On-channel blocking agent, 2-amino-4-phosphonobutyric acid (APB), Schiller (1984) showed the On-response in the surround of lateral geniculate nucleus (LGN) Off-cells in the Rhesus monkey is not affected by APB. This demonstrates that the On- and Off-channels remain independent up to and including the level of the LGN. However, in cortical complex cells APB blocked the responses to moving dark/light (Off) but not light/dark (On) edges, suggesting that the On- and Off-pathways converge at this level (Schiller, 1992). Behaviorally, the detection of light increments but not light decrements was severely impaired after injection of APB into the vitreous of the monkey (Schiller et al., 1986). The responses of DS cells in the middle temporal area (Zeki, 1974) and the medial superior temporal area (MST) of monkeys (Tanaka and Saito, 1989) have also been shown to be independent of contrast polarity, consistent with the notion that the On- and Off-pathways have already converged by this stage (see Section 2.3). For humans, Edwards and Badcock (1994) showed that psychophysical performance on a task requiring the global integration of local motion signals was similarly independent of contrast polarity. First, they found that the detection of a global motion signal defined by a set of luminance increment (On) dots moving in a common direction was impaired equally by the addition of randomly moving noise dots of either contrast polarity. Second, they found sub-threshold summation for global motion signals carried by a mixture of luminance increment (On) and luminance decrement dots (Off). Thus, it seems that the inputs to individual motion detectors in the human visual system preserve contrast-polarity, as evidenced by the reverse-phi phenomenon (Anstis, 1970; Anstis and Rogers, 1975), but that local motion information from these detectors is integrated in a manner independent of the contrast-polarity of the original image signals (Edwards and Badcock, 1994) Temporal characteristics of pre-filters If a motion detector received its inputs directly from photoreceptors without any temporal filtering (Fig. 3), the motion detector output would be modulated by motion but it would also respond strongly to a stationary image. Temporally band-pass filtering the image removes ongoing brightness signals so that only changes in contrast enter the detectors (Srinivasan et al., 1982). Temporal band-pass filtering can be implemented in a biological system by neurons with responses that are phasic. The impulse responses of such neurons, defined as the response to a brief flash, consist of an initial excitatory phase followed by an inhibitory period. The delayed temporal inhibition suppresses the sustained response that would otherwise be generated by a steady light. Neurons of this type respond primarily to changes in light intensity while constant intensity light produces virtually no response. Motion detectors receiving input from such band-pass filters will be tuned to respond selectively to temporal variations in the image rather than its unchanging components. This selectivity comes entirely through pre-filtering strategies. It is also possible to reduce the response of the subsequent motion processing mechanism to unchanging image components through subtraction of the outputs of motion detectors tuned to opposite directions of motion (see Fig. 1B). Ibbotson and Clifford (2001b) found evidence that the pre-filters to the motion detectors feeding the mammalian pretectal nucleus of the optic tract adaptively match their

5 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Fig. 3. Impulse response functions of (A and C) low-pass and (E and G) band-pass temporal filters and their corresponding temporal frequency response functions. The first-order low-pass filter in (A) has an exponentially decaying temporal impulse response function. The filter in (C) is third-order low-pass, equivalent to a cascade of three of first-order filters. The impulse response in (E) is the temporal derivative of that in (C), while that in (G) corresponds to modulating a third-order envelope with a sinusoid. (B and D) The filters with monophasic temporal impulse responses have low-pass temporal frequency response functions. Cascading serves to narrow the pass-band (compare D with B). (F and H) The filters with temporally modulating impulse response functions have band-pass temporal frequency response functions. response properties to the prevailing visual environment. In that study, the response to two-frame apparent motion was measured at a range of inter-stimulus intervals (ISIs) for stimulus contrasts of 20 and 80%. The stimulus consisted of two brief (10 ms) presentations of a sinusoidal grating separated by a variable ISI. Apparent motion was produced by displacing the second grating by one-fourth of a cycle relative to the first. For preferred-direction motion at the lower contrast, the response to the second frame of the apparent motion sequence was at least as large as the response to the first for all ISIs (Fig. 4). At the higher contrast, however, the response to the second frame was facilitated for short ISIs (10 50 ms) but attenuated for longer ISIs ( ms). For ISIs between 50 and 700 ms, the response to the second frame was actually facilitated by anti-preferred motion. This dependence of response sign on ISI duration is reminiscent of a range of apparent motion phenomena in human vision in which perceived direction of motion reverses for ISIs longer than around 60 ms (Shioiri and Cavanagh, 1990; Georgeson and Harris, 1990; Pantle and Turano, 1992). Ibbotson and Clifford (2001b) found that this behavior can be modeled by pooling the response of an array of elementary motion detectors whose inputs preserve signal polarity and whose pre-filter characteristics depend on stimulus contrast such that pre-filtering is temporally low-pass at low image contrasts and band-pass at high contrasts (Fig. 4). Such a coding strategy would tend to reduce the transmission of redundant information at high contrasts while maximizing signal strength at low contrasts (Srinivasan et al., 1982). Contrary to recent accounts (Strout et al., 1994; Johnston and Clifford, 1995a), these modeling results suggest that the dependence of perceived direction of apparent motion on ISI in human vision might reflect the implementation of a general purpose image coding strategy in early vision rather than a property particular to motion processing Temporal delay filtering A temporal asymmetry is a necessary component of any direction-selective motion detector (Borst and Egelhaaf, 1989). This could come in the form of a temporal delay filter (Reichardt, 1961) or in the form of a phase difference between two temporally modulated filters (Adelson and Bergen, 1985). Several attempts have been made to characterize the delay filters in biological motion detectors. Srinivasan (1983) used a stimulus in which a textured pattern was displaced a small distance in a single 5 ms frame. This impulsive image displacement was used to stimulate DS neurons in the insect optic lobe. The stimulus displacement produced an initial rapid increase in firing rate followed by an exponential decline in response level over the next 3 s. The response waveform was referred to as an impulse response (not to be confused with the common name for an action potential). Srinivasan (1983) was able to predict the response to continuous motion of the stimulus by convoluting the impulse response with the temporal profile of the

6 414 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Fig. 4. Simulation of responses to two-frame apparent motion. The response of the temporal pre-filters is the sum of excitatory and inhibitory components. (A) The excitatory component (solid line) of the temporal impulse response has a shorter time constant than the inhibitory component (dashed line). (B) The contrast response functions of the excitatory and inhibitory components are shifted relative to one another such that the excitatory component is responsive to lower contrasts. Simulated response of an array of correlation detectors with these pre-filters as a function of inter-stimulus interval duration at stimulus contrasts of: (C) 80%; (D) 20%. Preferred, anti-preferred and non-motion conditions are represented by solid, dashed and dotted lines, respectively. image motion. Several subsequent studies have used similar stimuli to record impulse responses of direction-selective neurons in insect and mammalian preparations (insects: de Ruyter van Steveninck et al., 1986; Maddess and Laughlin, 1985; Borst and Egelhaaf, 1987; mammals: Ibbotson and Mark, 1996). Although impulse responses proved useful in predicting the shape of response waveforms to continuous motion stimulation under certain stimulus conditions, they failed to predict the temporal frequency response functions (TFRFs) of the neurons (Harris et al., 1999). The TFRF characterizes the relationship between neuronal response magnitude and the rate of temporal modulation in the moving stimulus. The results of Harris et al. (1999) and those of previous studies (Zaagman et al., 1983; de Ruyter van Steveninck et al., 1986; Maddess, 1986; Borst and Egelhaaf, 1987; Ibbotson and Mark, 1996) demonstrate that the time course of decay of the impulse response is strongly dependent on stimulus history. It has been argued that this dependence might reflect adaptation at the level of the motion detector delay filter (de Ruyter van Steveninck et al., 1986; Clifford and Langley, 1996a; Clifford et al., 1997). However, subsequent studies have failed to show a corresponding adaptive shift in the preferred temporal frequency of the motion detectors (Ibbotson et al., 1998; Harris et al., 1999), which should remain inversely proportional to the length of the delay (Borst and Bahde, 1986; Egelhaaf and Borst, 1989; Clifford and Langley, 1996a; Clifford et al., 1997), as illustrated in Fig. 5. These later studies suggest adaptation at the pre-filter level as a more likely substrate of the variation in impulse response decay time constant as proposed by Maddess (1986), although Harris and O Carroll (2002) have recently shown that variation in the impulse response decay time constant can be modeled using fixed (non-adaptive) high-pass temporal pre-filters (see Section 4.5). Harris et al. (1999) attempted to characterize the impulse response of the motion detector delay filters feeding wide-field DS neurons in the insect optic lobes using an apparent motion stimulus. The stimulus consisted of two brief presentations of a sinusoidal grating in which the second presentation of the grating was displaced by one-fourth of a cycle in the preferred direction of the cell. Harris et al. (1999) plotted the magnitude of the response to the second flash as a function of the ISI between flashes. The resultant

7 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Fig. 5. Relationship between the decay time constant of the temporal impulse response and the peak of the temporal frequency response function. (A) Solid and dotted lines show impulse responses of first-order low-pass temporal filters with time constants in the ratio 2:1, such that the impulse response denoted by the solid line has the longer time constant. (B) Temporal frequency response functions of the same filters. The filter with the shorter time constant (dotted lines) has the temporal frequency response function that peaks at the higher temporal frequency. For time constants in the ratio 2:1, the peak temporal frequencies are in the ratio 1:2. response-isi functions increased rapidly for ISIs up to around 25 ms then decreased back to the size of the response produced by a single flash for ISIs of 200 ms or more. Harris et al. (1999) argued that, if the grating presentations can be considered as impulsive, the response-isi function is equivalent to the impulse response of the delay filter. Subsequently, Ibbotson and Clifford (2001b) measured response-isi functions for wide-field DS neurons in the mammalian pretectum to the apparent motion stimulus developed by Harris et al. (1999). Using computer simulations, Ibbotson and Clifford (2001b) demonstrated that the response-isi function is, in fact, heavily dependent on the pre-filtering of signals prior to the motion detector. Only by incorporating temporal pre-filtering into their model were Ibbotson and Clifford (2001b) able to relate the response-isi functions of neurons in the mammalian pretectal NOT to their TFRFs. When this was done, it was often possible to predict the TFRF and peri-stimulus time histogram (PSTH) of a given cell from its response-isi function to a reasonable degree of accuracy. The latter study shows that, to measure the temporal properties of the motion detector delay filter, it must be considered not in isolation but as part of a filter cascade that includes the temporal pre-filters Non-linear interactions A linear combination of adjacent samples of the image can produce a difference in response modulation for the two directions of motion (Fig. 6; Jagadeesh et al., 1993, 1997) but will not produce a directional time-averaged response (Watson and Ahumada, 1985). To produce direction-selective time-averaged outputs, a non-linearity is required. As a consistent directional time-averaged output is essential to signal the direction of motion, models that include some form of non-linearity are used to model the responses of direction-selective neurons. The nature of the non-linear stage has received much attention from theoreticians and several models have been proposed (e.g. Barlow and Levick, 1965; Adelson and Bergen, 1985; van Fig. 6. (Top) Space time plot of a sinusoidal grating stimulus drifting at constant velocity. (Middle) Variation about the mean level (dotted line) of the image signal at positions x1 and x2. The two signals have the same mean level, amplitude and temporal frequency but differ in temporal phase. (Bottom) Delaying one signal relative to the other shifts their relative temporal phase. For motion in the preferred direction, this results in constructive interference between the two temporal signals and a large variation about the mean level. Motion in the anti-preferred direction puts the two signals close to opposite temporal phases such that, in the sum, the variation about the mean level is small. Linear motion detectors of this kind are directional in their degree of response modulation but not in their mean response level.

8 416 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Santen and Sperling, 1985; Egelhaaf et al., 1989; Amthor and Grzywacz, 1993; Johnston et al., 1992). We will now describe the types of non-linear interactions that have been considered and the evidence for their existence in biological systems Facilitation Conceptually, the simplest possible non-linear process is a direct facilitatory interaction such as a multiplication (Fig. 1B). This type of interaction was first proposed by Hassenstein and Reichardt (1956) in their Correlation model, subsequently referred to as the Reichardt detector, to account for behavioral data from the Chlorophanus beetle. While there is no evidence to suggest that multiplication occurs at a single synapse (Egelhaaf and Borst, 1992), non-linear facilitation may arise through initial linear combination of signals (Watson and Ahumada, 1985) followed by a non-linear operation such as squaring (Adelson and Bergen, 1985), rectification (Mizunami, 1990), or thresholding (Jagadeesh et al., 1997). For example, Adelson and Bergen (1985) proposed an Energy model where signals from adjacent locations are summed or subtracted. Such operations produce space time oriented linear filters (STOLFs), i.e. filters in which response latency varies systematically with spatial position. Using a prime to denote a delayed signal, the combinations of inputs from adjacent locations, A and B, that produce space time oriented linear filters are: A B, A + B, B+A and B A. As the combination is linear, a space time oriented linear filter will produce a response if A or B is stimulated alone, with the latency of response depending on the spatial position. Temporal coincidence is not required to generate a response from such filters. Conversely, multiplication will only give a response if both input channels are stimulated with an appropriate time interval. Thus, while direct (multiplication) and indirect (linear combination followed by squaring) mechanisms detect oriented structure in space time, only the indirect models contain space time oriented linear filters (Fig. 7). The responses of such filters depend both on the direction-of-motion and the phase of the image signal. The Energy model combines the squared outputs of these filters to produce directional responses, referred to as motion energy. The response of the Energy model at this stage depends on temporal coincidence and is independent of the phase of the image signal. The responses of small-field DS neurons in the optic lobes of insects to moving sinusoidal gratings oscillate around the mean response level at the fundamental and second harmonic frequencies of the stimulus temporal frequency (DeVoe, 1980). Similar fundamental and second harmonic responses can be observed in the responses of wide-field neurons if the gratings are presented in restricted areas of a cell s visual field (Egelhaaf et al., 1989; Ibbotson et al., 1991). The amplitude of oscillation at higher-order harmonics was found to be negligible, implying that a second-order (quadratic) non-linear interaction occurs in the elementary motion detectors feeding into the neurons. Similar experi- Fig. 7. Space time oriented linear filters (STOLFs). Space time plots of STOLFs preferring motion (A) to the right, (B) to the left. The preferred speed and direction of each STOLF corresponds to its orientation in space time. (C) Rightwards motion of a white bar across the gray background produces a space time trajectory matched to the space time receptive field structure of the STOLF preferring rightwards motion. (D) Rightwards motion of a contrast-reversing bar produces a space time trajectory better matched to the STOLF preferring leftwards motion, consistent with the perception of reverse motion. ments on neurons in the mammalian nucleus of the optic tract also revealed mainly fundamental and second harmonic responses (Ibbotson et al., 1994), suggesting the operation of a quadratic non-linearity in the motion detectors feeding into the NOT. To test this suggestion Ibbotson et al. (1999); Ibbotson (2000) measured the responses of neurons in the NOT using apparent motion stimuli consisting of successive presentations of identical contrast changes in two adjacent bars. Increasing the contrast of the bars increased response magnitudes in an approximately quadratic fashion up to contrasts of 25%, supporting the notion of a facilitatory non-linearity (Fig. 8). However, values beyond that contrast led to a saturating response function, such that the overall contrast response function had a sigmoidal appearance. These data emphasize that it is important to consider the effect of other non-linearities in the system, such as spike generating mechanisms, in determining the measured neuronal response, especially at high stimulus contrasts. Since the multiplication stage of the Reichardt model and the squaring stage of the Energy model are both forms of quadratic non-linearity, it is difficult to distinguish direct and indirect models at the physiological and behavioral levels. For example, in their mathematically perfect forms, the Energy and Reichardt models produce identical outputs (van Santen and Sperling, 1985; Emerson et al., 1992), since AB = [(A + B ) 2 (A B ) 2 ]/4. However, biological systems operating according to these two principles are not indistinguishable at all stages. Emerson et al. (1992) recorded the responses of complex cells in cat striate cortex thought to be elements within the sub-units of the Energy model. By

9 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Fig. 8. Response vs. contrast function for an NOT neuron. Filled circles show the response to preferred direction apparent motion, stars show the peak non-motion response, and open symbols show the response to anti-preferred apparent motion. The fitted curves are the best-fit quadratic functions for contrasts up to 25%. recording the response to two bars displaced in space and time and subtracting off the responses to the bars presented individually, Emerson et al. (1992) were able to calculate the non-linear interaction of the two bars as a function of their spatial and temporal displacement. The data revealed space time oriented two-bar interaction fields that could be simulated by spatially integrating the outputs of motion energy sub-units but could not be simulated by any stage of the Reichardt detector Inhibition In the Reichardt and Energy models, the essential non-linearity is facilitatory. To explain the responses of On Off-DS retinal ganglion cells in the rabbit, Barlow and Levick (1965) proposed two mechanisms, a facilitatory and an inhibitory model. The facilitatory model was conceptually similar to a single sub-unit in the Reichardt detector, while under the inhibitory model responses in one direction were selectively inhibited. The inhibitory scheme has been modeled at the synaptic level using shunting inhibition (Torre and Poggio, 1978; Koch et al., 1983). Inhibition is generated by increasing the membrane conductance of a neuron and shunting incoming currents out from the cell. This interaction is division-like because shunting inhibition divides the excitatory currents by the membrane conductance. The inhibitory model was tested by recording the responses of On Off-DS cells in the rabbit retina to apparent motion in the preferred (Amthor and Grzywacz, 1993) and null directions (Grzywacz and Amthor, 1993). While preferred direction excitation produced linear facilitation, null direction inhibition appeared to be characteristic of Fig. 9. Responses to apparent motion of On Off-DS cells in the rabbit retina. In the following account, the stimulus consists of two adjacent bar stimuli (labeled slit-a and -B) placed in the cell s receptive field and oriented perpendicular to the cell s preferred direction. The cell s preferred direction is from slit-a to -B. In all cases, the expected response to the first slit has been subtracted. (A) Responses to apparent motion in the null-direction. The upper curve shows the average response vs. contrast function generated when slit-a was presented alone (curve marked 0%). Other curves show the functions produced when the motion sequence was slit-b then slit-a (slit-b contrasts are shown alongside the respective curves). As the contrast of slit-b increased, the response vs. contrast function produced by subsequent presentation of slit-a was more strongly attenuated compared to stimulation of slit-a alone. (B) Response vs. contrast curves for apparent motion in the preferred direction. The dashed line shows the response function generated by stimulating slit-b alone. Other curves show functions obtained during apparent motion from slit-a to -B (slit-a contrasts were 0, 10, 20 and 30%, as illustrated). Increasing the contrast of slit-a additively enhances the response elicited by the second slit, so curves are shifted upwards in parallel. Adapted from Fig. 10 of Grzywacz and Amthor (1993) and Fig. 10 of Amthor and Grzywacz (1993). a non-linear division-like process similar to that expected from the inhibitory scheme (Fig. 9). Subsequently, Holt and Koch (1997) have shown that shunting inhibition has a divisive effect only on sub-threshold excitatory post-synaptic potential amplitudes. Shunting inhibition actually has a subtractive effect on the firing rate in most circumstances because the spiking mechanism appears to clamp the somatic membrane potential to a level above the resting potential. Consequently, the current through the shunting conductance is independent of the firing rate, which leads to a subtractive rather than a divisive effect. Holt and Koch (1997) suggest that observation of divisive inhibition in the spiking properties of cortical

10 418 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) neurons might be due to effects generated at the network rather than the synaptic level. Given the complex circuitry that forms the input to retinal ganglion cells in the rabbit (see Section 2.1 and Fig. 12), such network effects might underlie the division-like inhibition observed by Grzywacz and Amthor (1993). Direction-selective responses have been driven in rabbit On Off-DS ganglion cells by edges of light moving only 1.1 m (26 of visual angle) across the retina (Grzywacz et al., 1994). This distance is smaller than the spacing between rabbit photoreceptors, which is approximately 1.9 m or46 (Young and Vaney, 1991). It is suggested that this directional hyperacuity is the result of low-noise high-gain signal transmission from the photoreceptors to the ganglion cells. Moreover, the result suggests that directional selectivity can be generated in small portions of the dendritic processes of ganglion cells and does not require a whole cell mechanism. The intracellular mechanisms leading to the generation of direction-selective responses in rabbit On Off-DS cells have been studied using patch clamp recording (Taylor et al., 2000). Taylor et al. first showed that movement in the cell s preferred direction caused a greater excitatory current to enter the cell s dendrites. They then voltage-clamped the dendritic membrane at 70 and 30 mv and recorded the synaptic currents produced by a moving bar. The difference between the synaptic currents generated by preferred and null direction motion was more pronounced when the cell was more depolarized ( 30 mv) and was predominantly caused by an increase in inhibition for null direction motion. When the intracellular concentration of chloride was equilibrated to the extracellular level, making the reversal potential of -aminobutyric acid A (GABA A ) receptor-mediated inhibition equal to that of excitation, this difference disappeared. From this, it was concluded that a major component of the direction selectivity of DS retinal cells in the rabbit is generated by null-direction inhibition acting post-synaptically to the ganglion cell dendrites (Taylor et al., 2000). However, contrary to the findings for rabbit On Off-DS cells, Borg-Graham (2001) found that DS retinal ganglion cells in the turtle were not the site of the non-linear interaction and that direction-selective coding probably occurred earlier in the visual system. Details of motion processing prior to retinal ganglion cells in the turtle are given in Section 3.1 (DeVoe et al., 1989). Borg-Graham (2001) questioned the assumption that chloride loading simply transforms all inhibitory inputs to excitatory ones while leaving the original excitatory inputs unchanged. He suggested that the effects of high intracellular chloride might be more complex, casting doubt on the interpretation of ganglion cell dendrites as the site of the non-linear interaction in the rabbit retina Speed-tuned motion detectors In a spatiotemporal frequency response profile, where neuronal response is plotted as a function of spatial frequency on the abscissa and temporal frequency on the ordinate, there are diagonally oriented ridges of peak sensitivity Fig. 10. Temporal frequency tuning vs. speed tuning. (A) Schematic spatio-temporal frequency response function for a speed-tuned neuron. Iso-response contours have their major axis lying along an iso-speed line (dotted). For all spatial frequencies, the peak response is obtained at the same speed. (B) Schematic spatio-temporal frequency response function for a temporal frequency-tuned neuron. The spatio-temporal frequency response function is the product of separable spatial and temporal frequency response functions. Iso-response contours have their major axis parallel to the spatial or temporal frequency axis. For all spatial frequencies, the peak response is obtained at the same temporal frequency. when a cell is speed tuned (Fig. 10A). The orientation of the ridge corresponds to a particular speed of image motion. Alternatively, if the cell is not tuned to speed but rather to specific spatial and temporal frequencies, as in the final output of the Reichardt and Energy models, we would expect a response profile with elliptical contours whose major axes are parallel to the spatial and temporal frequency axes (Fig. 10B). For a diagonally oriented ridge, space and time are inseparable, meaning that the cell s response profile is not simply the product of separate spatial and temporal filters. An alternative theoretical approach to motion detection is provided by the gradient model (Fennema and Thompson, 1978; Horn and Schunk, 1981; Srinivasan, 1990; Johnston et al., 1992). Gradient-based approaches use filters that take fuzzy derivatives (Koenderink and van Doorn, 1987), blurring and differentiating the image, and combine the filter outputs as a quotient of temporal and spatial derivatives to estimate velocity (Fennema and Thompson, 1978; Horn and Schunk, 1981; Johnston et al., 1992, 1999; Johnston and Clifford, 1995a). Under this scheme, motion is computed from the ratio of temporal and spatial frequency-tuned channels. The spatial and temporal frequency-tuned mechanisms in the gradient model are non-directional but their combination at the division stage produces a directional, speed-tuned response. The division operation can be thought of as a form of inhibitory non-linearity. Directionality and speed-tuning emerge from the gradient scheme at a later stage than temporal frequency tuning. This is in contrast to the Reichardt model where speed-tuning at the sub-unit stage is replaced by temporal frequency tuning at the motion opponent stage (Zanker et al., 1999). Although a speed-tuned signal is available from the output of a Reichardt sub-unit, the signal is superimposed on a large non-motion related signal making it an impractical

11 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) site to extract speed information. The Energy model contains space time oriented linear filters (Fig. 7) that are both directional and temporal frequency-tuned (Emerson et al., 1992). Several models have been proposed to show how speed could be extracted by analyzing the distributed output of motion-energy sub-units tuned to selected spatial and temporal frequencies (Heeger, 1987; Grzywacz and Yuille, 1990; Simoncelli and Heeger, 1998). However, Ascher and Grzywacz (2000) point out that these models are often based on rather theoretical filters that do not fit with existing biological data. Ascher and Grzywacz (2000) present a Bayesian model for the measurement of visual velocity that allows the estimation of retinal velocity with more realistic assumptions about the form of the spatial and temporal filters. Importantly, the model is consistent with observed aspects of speed perception such as the dependence of perceived speed on contrast (Thompson, 1982). Speed-tuning is a common property of direction-selective neurons in primate middle temporal area (Rodman and Albright, 1987; Perrone and Thiele, 2001, 2002). A small percentage of direction-selective neurons have also been identified as speed-tuned in the mammalian pretectal nucleus of the optic tract (Ibbotson and Price, 2001) and in its avian homologue, the pretectal lentiformis mesencephali (Wylie and Crowder, 2000). Spatiotemporal receptive field profiles in primate primary visual cortex (V1) are not tuned to image speed but rather to specific spatial and temporal frequencies (Foster et al., 1985). Some V1 neurons have low-pass (sustained) response profiles while others are band-pass (transient) (Foster et al., 1985; Hawken et al., 1996). Perrone and Thiele (2002) suggest that V1 neurons with separable response functions provide the input to speed-tuned neurons in the middle temporal area. They provide a model, the weighted intersection model (WIM), of how the visual system extracts speed independent of spatial frequency. The WIM predicts that the maximum output of a speed-tuned middle temporal area (V5) (MT) neuron occurs whenever it receives equal input from sustained and transient V1 neurons. The WIM is related to gradient schemes of motion detection in that the response of its model MT neurons depends on the ratio of sustained and transient input. However, while gradient schemes typically compute image speed directly from the ratio of the responses of transient and sustained mechanisms, the corresponding stage of the WIM model shows band-pass speed tuning consistent with the properties of MT neurons. Much has been made in psychophysical circles of the mathematical equivalence of the various classes of motion detection scheme (e.g. van Santen and Sperling, 1985; Adelson and Bergen, 1986). For example, a least squares gradient estimator of velocity based on Gaussian derivative filters can be redescribed as an opponent Energy model based on filters oriented diagonally in space time (Adelson and Bergen, 1986), while Energy and Correlation models with the same constituent filters effectively perform the same computations in a different order (Emerson et al., 1992). To identify decisively how image motion is computed in a particular biological system it is necessary to investigate the component elements of the system (see Section 3). The differences between models in terms of directionality and temporal frequency tuning mean that in principle they are distinguishable physiologically. Of course, one problem with such an enterprise is knowing at which stage of the functional motion processing hierarchy you are recording. However, given that gradient models have been shown to have considerable predictive power in terms of the psychophysics of motion perception in humans (Johnston and Clifford, 1995a,b; Benton et al., 2000, 2001), more work is warranted to investigate how such models might map onto the underlying physiology. For example, Johnston et al. (1992) have shown that stages of their Multi-channel Gradient model of motion perception respond to drifting sine wave gratings in a manner resembling that of some cortical simple and complex cells in terms of directionality and phase independence. 3. Evidence for the cellular mechanisms of motion detection 3.1. Retinal motion detectors in vertebrates The vertebrate retina contains all of the sequential stages and lateral connections required for motion detection (Fig. 11; Dowling, 1979), although these may not be fully utilized in all species. The pathway begins with the photoreceptors, then a layer of bipolar cells and finally the RGCs, which form the output of the retina (Fig. 11). At the interface between the photoreceptors and the bipolar cells is a layer containing horizontal cells that provide the substrate for lateral interactions between neighboring areas of the visual field. The horizontal cells provide the substrate for the lateral inhibition that generates center-surround interactions and concentric receptive fields in bipolar cells and RGCs (e.g. Kuffler, 1953). More lateral interconnections are provided by amacrine cells, which form a second horizontal layer at the interface between the bipolar and ganglion cells. As outlined in Section 2.2.1, early recordings from the rabbit retina revealed two types of direction-selective RGCs: On- and On Off-DS RGCs (Barlow et al., 1964; Barlow and Levick, 1965). Both cell types were direction-selective but the On-cells responded only to the movement of bright edges while the On Off-cells responded to the movement of both bright and dark edges. The discovery that some RGCs were directional provided an excellent opportunity to use the rabbit retina as a model system to investigate the cellular mechanism responsible for direction-selectivity in a biological system (for review see Vaney et al., 2001). The On Off-DS RGCs have a bistratified morphology (Amthor et al., 1984, 1989; Oyster et al., 1993). The outer dendritic stratum receives input from Off-center neurons (depolarized by brightness decrements) and the inner dendritic stratum receives input from On-center cells (depolarized

12 420 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Fig. 11. The retina has three nuclear layers: The outer nuclear layer (photoreceptors), the inner nuclear layer (bipolar, horizontal and amacrine cells) and the ganglion cell layer (ganglions). Between the inner and outer nuclear layers is the outer plexiform layer where lateral connections are formed between photoreceptors, bipolar cells and horizontal cell processes. Between the inner nuclear layer and the ganglion cell layer is the inner plexiform layer where lateral connections are formed between bipolar, amacrine and ganglion cells. Information flows from photoreceptors to ganglion cells but there are also many lateral interactions. by brightness increments). The dendrites of the On Off-DS cells co-stratify with cholinergic (starburst) amacrine cells (Vaney et al., 1989), which provide excitatory inputs to the ganglion cells (Masland and Ames, 1976; Ariel and Daw, 1982). The dendrites of the On-DS RGCs are monostratified and reside in the same inner dendritic stratum as the On-dendrites of the On Off-DS RGCs (Amthor et al., 1989). The main difference between the dendrites of the two RGC types appears to be that the On DS cells have receptive fields that are approximately three times wider than the On Off-DS cells (Pu and Amthor, 1990). The On Off-DS cells do not have anatomical features that correlate with the cells preferred response directions, but they do form a very dense tiling mosaic across the retina where cells of the same type establish non-overlapping spatial domains (Oyster et al., 1993; Amthor and Oyster, 1995). How do the structural elements discussed above fit into the theoretical approaches discussed in Section 2? A series of experiments has shown that amacrine cells appear to provide the neural substrate for some of the lateral interactions required for motion detection (Fig. 12). More specifically, starburst amacrine cells provide the substrate to potentiate the responses of ganglion cells to motion in all directions and possibly to generate preferred direction facilitation (Grzywacz and Amthor, 1993; He and Masland, 1997). Significant motion facilitation can arise from inputs well outside the region of retina occupied by the dendritic arborizations, which corresponds to the classical receptive Fig. 12. Schematic of the neural circuitry thought to underlie direction-selectivity in the rabbit retina (adapted from Vaney et al., 2001). DS RGC, direction-selective retinal ganglion cell; Bip, bipolar cell; SA, starburst amacrine cell; GA, GABAergic amacrine cell. Note that there are many cones feeding into each DS RGC, with much summation en route. This could give the impression that direction-selectivity is generated quite coarsely within the receptive fields of the ganglion cells. However, edges of light moving only 1.1 m, which is smaller than the inter-photoreceptor distance, can generate directional responses (Grzywacz et al., 1994). This result suggests low-noise high-gain signal transmission from the photoreceptors to the ganglion cells and the generation of direction selectivity in small portions of the dendritic processes of ganglion cells (see Section 2.4.2).

13 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) field, of the On Off-RGCs for preferred direction motion but not for the anti-preferred direction (Amthor et al., 1996). Anti-preferred inhibition appears to arise from a different set of GABAergic amacrine cells, which again form synapses onto the DS RGCs (Fig. 12; Grzywacz et al., 1997; Massey et al., 1997). Consequently, motion in the preferred and anti-preferred directions is actually driven by different systems rather than identical systems with mirror symmetric directional tuning, as predicted by theoretical motion detectors such as the Reichardt and Energy models. Moreover, Section 2.3 has already shown that the biophysical mechanisms underlying the response to motion in the preferred and anti-preferred directions are different (Amthor and Grzywacz, 1993; Grzywacz and Amthor, 1993), i.e. preferred direction motion produces linear facilitation while null direction motion produces non-linear inhibition (Fig. 9). The rabbit retina has provided much information about the cellular mechanisms of retinal motion detection but retinas in other vertebrate species have also yielded important results. Perhaps the most thoroughly studied example is the turtle retina (DeVoe et al., 1989). Although many results from the turtle retina provide evidence of a similar mechanism to the rabbit (Marchiafava, 1979; Ariel and Adolph, 1985; Kittila and Granda, 1994; Smith et al., 1996), intriguing differences have also been identified. Perhaps the most obvious is that the turtle retina has On Off, On- and Off-DS cells, while the rabbit does not have the Off-specific type (Jensen and DeVoe, 1983). DeVoe et al. (1989) showed in the turtle that 33% of retinal ganglion cells were direction-selective. They also showed that 37% of amacrine cells and 42% of bipolar cells were directional. The retinal ganglion cells were fully direction-selective, giving spikes in one direction and no spikes in the opposite direction, suggesting strong non-linear mechanisms. In bipolar and amacrine cells, post-synaptic potentials were larger for movement in one direction than the opposite but the cells were not motion opponent. A small number of directional turtle horizontal cells have been identified (Adolph, 1988; DeVoe et al., 1989). They show far smaller changes in amplitude between opposite directions of motion than are observed in bipolar, amacrine or ganglion cells. Evidence for cholinergic and GABAergic processes in both the inner and outer plexiform layer, combined with evidence of directional tuning even in the inner segments of a small number of cone-type photoreceptors (Carras and DeVoe, 1991), suggests that at least some directional coding occurs very early in the distal retina of the turtle (Criswell and Brandon, 1992). It should be noted that the directionally biased responses recorded in the cones of the turtle retina, as with the bipolar and amacrine cells, did not have the full directional properties associated with DS retinal ganglion cells (Carras and DeVoe, 1991). Rather, responses to motion in one direction were simply larger than those to other directions (see Fig. 6), which corresponds to the expectations of the early linear stages of certain theoretical motion detectors such as the Energy model (Adelson and Bergen, 1985; Emerson et al., 1992). Anatomical evidence shows that some turtle cones have asymmetrically radiating telodendria (Ohtsuka and Kawamata, 1990), which could provide the spatial asymmetry required for early motion processing between photoreceptors at the level of the retina s outer plexiform layer. In summary, the rabbit retina has revealed a neural architecture that contains bundles of DS ganglion cells that are innervated by amacrine and bipolar terminals. This organization provides all of the wiring required for calculating the direction of image motion. Perhaps most interesting is the finding that the actual structure of the rabbit s retinal motion detectors does not correspond exactly with any of the theoretical models. Rather, evolution has developed a system that utilizes all of the theoretical concepts described in the models but is organized in a way not predicted by theoreticians. This is certainly a lesson for physiologists who attempt to find exact correlates of models in neural tissue. Rather, a broad approach is required that is guided but not driven by the expectations of theoretical models. Another interesting observation from the turtle retina is that the beginnings of directional tuning may occur as early as the distal retina. Moreover, it is important to look for directionally biased responses rather than motion-opponent responses when searching for the input structures of biological motion detectors Sub-cortical motion processing Although the lateral geniculate nucleus shows some directional effects (Lee et al., 1979; Thompson et al., 1994; White et al., 2001), the most striking regions of the sub-cortical mammalian brain in terms of direction selectivity are the pretectum and accessory optic system (AOS) (e.g. Simpson, 1984). The pretectum contains the NOT and the AOS consists of three nuclei: the lateral, medial and dorsal terminal nuclei (LTN, MTN, DTN). In all of these nuclei, the most commonly encountered cells are those that respond in a highly direction-selective (motion-opponent) manner to the movement of large regions of the visual scene. In most cases, the cells give motion opponent responses in which motion in one direction excites the cell while motion in the opposite direction inhibits the cell s spontaneous activity (Collewijn, 1975a; Hoffmann, 1989; Ibbotson et al., 1994). The NOT and AOS are connected to the motor system that controls stabilizing eye movements such as optokinetic nystagmus (Collewijn, 1975b; Schiff et al., 1988; Belknap and McCrea, 1988). The NOT and the nuclei of the AOS receive direct input from the retina in all species studied (e.g. cat: Ballas and Hoffmann, 1985; monkey: Telkes et al., 2000). In some animals, such as monkey and cat, there is also an indirect input from the visual cortex (cat: Schoppmann, 1981; monkey: Distler et al., 2002) but in other species, such as the marsupial opossum (Pereira et al., 2000) and wallaby (Ibbotson et al., 2002), there is no cortical input.

14 422 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) The motion opponent character of the responses in these nuclei suggests that the action potentials are generated in the cells after the final subtraction stage of the motion processing mechanism, as outlined in Section 2. This observation leaves the possibility that the final subtraction phase actually occurs in the dendrites of the NOT and AOS neurons prior to the site of spike generation. The directional responses in the NOT of the wallaby, which does not receive input from the cortex (Ibbotson et al., 2002), compare very well with the final subtraction stage of both the Energy and Reichardt models (Ibbotson and Clifford, 2001a,b; Ibbotson et al., 1994). It has also been established that the fundamental non-linearity in the motion processing mechanism is quadratic, as predicted by both models (Section 2.4.1; Ibbotson et al., 1999). The spatiotemporal response properties of neurons in the sub-cortical motion processing areas of the avian brain suggest that similar motion detector mechanisms are in operation (Wylie and Crowder, 2000). Indeed, the spatiotemporal tuning of cells in the avian and mammalian brain are organized in a very similar fashion across the cell population, suggesting that the visual environment during head and eye movements molds the spatiotemporal properties of the neurons across widely separated phyla (Ibbotson and Price, 2001). The retinal ganglion cells that provide the input to the NOT and AOS in vertebrates are generally slowly conducting ganglion cells with small- to medium-sized cell bodies. These are so-called specialized cells in primates (Telkes et al., 2000) and W-cells in cats, rabbits and rats (Ballas et al., 1981; Pu and Amthor, 1990; Kato et al., 1992; Rodieck and Watanabe, 1993). It is known that these cells are direction-selective in cats (Hoffmann and Stone, 1985), rabbits (Oyster et al., 1972) and turtles (Rosenberg and Ariel, 1991). Ilg and Hoffmann (1993) found that most cortical cells that could be stimulated anti-dromically from the NOT were strongly directional. Therefore, the cortical input in primates is also already direction-selective, as evidenced by the strong input from directional fibers from the extrastriate motion processing areas MT and MST (Hoffmann et al., 2002). Presumably, the role of the NOT neurons is to summate the inputs from directional cells to generate selective responses to large field stimulation. Moreover, as many cells from the visual cortex are not motion opponent, the NOT neurons might provide the neural substrate for the final subtraction phase to produce motion opponency. The evidence suggests that the cellular mechanisms of directional motion detection occur primarily before the NOT or AOS. Most directional properties probably arise in the retina but in certain species there is an additional input from higher visual centers in the cortex Cortical motion processing The visual cortex is one of the most heavily studied areas of the brain. The majority of work on cortical motion processing has been conducted on cats (Hubel and Wiesel, 1962, 1965) and primates (Hubel and Wiesel, 1968; Foster et al., 1985). However, the physiology of cortical neurons has also been studied in a number of other mammals such as the rabbit (Murphy and Berman, 1979), opossum (Rocha-Miranda et al., 1973) and wallaby (Ibbotson and Mark, in press). The general finding in all the species mentioned is that directional responses are common in the primary visual cortex (referred to as area 17 or area V1). Other areas in the visual cortex are known to specialize in coding motion information, notably the middle temporal area (MT or V5) in primates (Dubner and Zeki, 1971) and the posteromedial lateral supersylvian area (PMLS) in cats (Blakemore and Zumbroich, 1987). Cells fall into three main categories: (1) non-directional; (2) directionally biased (non-opponent); and (3) motion opponent. Cells in the first category are not directional but may be quite strongly orientation tuned, e.g. they might respond strongly to vertically oriented gratings but not to horizontal gratings (Mazer et al., 2002). Orientation tuned cells usually give their best responses when an oriented bar or grating is moved back and forth along an axis perpendicular to the preferred orientation. The directionally biased cells tend to be orientation tuned but the response to motion in one direction along the preferred motion axis (which is perpendicular to the preferred orientation axis) is stronger than the response in the opposite direction (Henry et al., 1974). Neurons with motion opponent properties respond strongly in the preferred direction and are inhibited by motion in the opposite direction. It has recently been reported that, at high speeds, many neurons in the primary visual cortex of cat and monkey appear to be selective for motion parallel rather than perpendicular to their preferred orientation (Geisler et al., 2001). This has been taken as support for the hypothesis that spatial streaks caused by motion smear in the image can also be used as a cue in the perception of motion (Geisler, 1999). The majority of cells in the LGN, which is the primary relay between the retina and cortex, have responses that are similar to those of center-surround retinal ganglion cells (Hubel and Wiesel, 1961). Some LGN cells show weak orientation tuning and some directional effects (Lee et al., 1979; Thompson et al., 1994; White et al., 2001). However, as a general statement, the LGN contains cells that have much weaker orientation and directional preferences than the cortex. It is therefore reasonable to suggest that V1, or at least the interface between the LGN and V1, is a good place to look for the cellular basis of motion detection in the geniculo-cortical pathway. However, before discussing the evidence from the LGN and V1 themselves, it is worth considering evidence from higher areas of the cortex that receive input from V1. The most striking region of the primate brain in terms of motion processing is the middle temporal area, MT or V5 (Dubner and Zeki, 1971). The majority of cells in MT are direction-selective but it does not appear to extract local motion information itself but rather receives directional signals from earlier stages in the visual system (Livingstone

15 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) et al., 2001). Lesions of V1 in primates greatly reduce the prevalence of DS cells in MT but do not totally abolish the phenomenon (Rodman et al., 1989; Girard et al., 1992). MT also appears to receive directional inputs directly from sub-cortical structures such as the colliculus and pulvinar (Rodman et al., 1990; Bender, 1982; Beckers and Zeki, 1995). Movshon and Newsome (1996) recorded from V1 neurons that could be anti-dromically activated by electrical stimulation of MT. They found that these cells were already directionally biased. The evidence therefore points towards V1 as a major location for motion computation. It has been proposed that directional V1 cells are local motion energy filters (Adelson and Bergen, 1985; Heeger, 1987; Grzywacz and Yuille, 1990; DeValois et al., 2000). As outlined in Section 2, there is evidence to suggest that certain simple and complex cells in cat V1 have directional properties that are similar to those predicted by stages of the Energy model (Emerson et al., 1992; Emerson, 1997; Emerson and Huang, 1997). Livingstone et al. (2001) recorded from MT neurons in primates and showed that directionality occurred for sequential presentation of stimuli less than 1/10 of a degree apart. This distance is far smaller than the receptive field sizes of most directional V1 cells. The implication is that interactions are most probably occurring between sub-regions or -units within the receptive fields of V1 cells (Fig. 13). It was also established that responses of MT Fig. 13. Schematic of the flow of information from lateral geniculate nucleus (LGN) to the middle temporal area (MT) via the primary visual cortex (V1). The LGN has lagged and non-lagged cells that feed into V1 cells. In primate V1, non-directional lagged and unlagged cells have been identified (DeValois et al., 2000). In cat, cells with lagged and unlagged zones have been found (Saul and Humphrey, 1992). In primates, it would appear that V1 cells then feed into other V1 neurons such as special complex cells. However, the special complex cells may also receive direct input from the LGN. By the time information reaches MT it has been processed so that very specific movement information is integrated together. cells to sequentially presented neighboring bars of opposite contrast (i.e. light bar then dark bar) produced inverted responses. This result, as discussed in Section 2.2, indicates that the polarities of the signals entering the motion detectors are preserved. Retinal ganglion cells, LGN neurons and simple cells in the visual cortex show inverted responses to opposite contrasts while complex cells in the cortex do not. Therefore, Livingstone et al. (2001) suggest that direction-selectivity is generated within or between geniculate inputs or simple cells. Movshon and Newsome (1996) found that the V1 neurons that projected directly to MT were of the special-complex type, i.e. they responded to a broad range of spatial and temporal frequencies and were sensitive to very low contrasts. It is probable that the special complex cells receive their input from directional simple cells or directly from LGN fibers or both (Fig. 13). Evidence suggests that the spatial separation between inputs arises from the differences in the receptive field locations of neighboring LGN or simple cells. DeValois et al. (2000) find that directional V1 cells in the macaque monkey get the inputs required for motion detection by combining signals from two identified sub-populations of non-directional cortical neurons. These sub-populations differ in the spatial phases of their receptive fields. That is, both cell types have On and Off zones but corresponding zones are spatially displaced with respect to each other. The two sub-populations also have distinct temporal properties: those with a slow monophasic temporal response and those with a fast biphasic temporal response. The fast biphasic cells cross over from one response phase to the reverse just as the monophasic cells reach their peak response. This 90 (quadrature) phase difference would make the two sub-populations of cells ideal building blocks for the Energy model. Where might be the origin of the temporal differences in the non-directional V1 sub-populations identified by DeValois et al. (2000)? One possibility is that temporal differences arise from LGN neurons (Fig. 13). In the macaque, parvocellular LGN cells are slow and largely monophasic while magnocellular LGN cells are fast and biphasic, leading DeValois et al. (2000) to suggest that the two non-directional cortical sub-populations they identify might receive their input from parvo and magno LGN cells, respectively. In the cat, X- and Y-relay cells in the LGN have been classified as lagged or non-lagged (Mastronarde, 1987; Mastronarde et al., 1991; Humphrey and Weller, 1988). When stimulating with sinusoidally luminance-modulated stimuli, lagged and non-lagged cells fire about a quarter of a cycle out of phase at low temporal frequencies (<4 Hz). Saul and Humphrey (1990) simulated the input of lagged and non-lagged cells onto cortical neurons and showed that the responses would be direction-selective at low temporal frequencies but would lose that directionality at higher frequencies (>4 Hz). Saul and Humphrey (1992) went on to use sinusoidally luminance-modulated stimuli in cortex to search for signs of lagged and non-lagged inputs to cortical

16 424 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) neurons. They found lagged and non-lagged zones within the receptive fields of the cortical neurons. The distribution of latency and absolute phase across the sample of cortical simple cells was similar to that found in the LGN. The similarity between cortex and LGN was greatest in the geniculate recipient layers of the cortex. In conclusion, the results from various mammals indicate that the cortex contains a plethora of cell types within the very general categories of simple and complex cells. The smallest image displacement that leads to a directional response is considerably smaller than the receptive field size of most simple cells, suggesting that the essential lateral interactions may occur between the terminals of LGN neurons. Subsequent processing progresses from linear summation of signals in some LGN neurons up to highly directional responses in the special complex cells that project to MT (in the case of primates). It is clear that far less is known at the cellular level in the cortex than in the retina. However, the general architecture of the cortical system is slowly being revealed Motion detectors in insect optic lobes As alluded to in Section 2, a great deal of information relating to motion detection has arisen from work on insects, including the classic work of Hassenstein and Reichardt (1956). It is interesting to look at the mechanisms of motion detection at the cellular level in insect optic lobes, which are made up of three neuropiles (Fig. 14). Starting just below the retina and working inwards towards the brain, these neuropiles are the lamina, medulla and lobula complex. The most thoroughly studied insect visual system is that found in flies (Douglass and Strausfeld, 2001) but the nervous sys- tems of other insects have also contributed to the field (e.g. bees: Ibbotson, 1991a; moths: Milde, 1993; locusts: Osorio, 1986). The lobula complex in flies is divided into two compartments. The most dorsal compartment is referred to as the lobula plate and it contains direction-selective neurons (reviewed by Hausen, 1993) and has been described as a tectum-like structure (Douglass and Strausfeld, 2001). The neurons in the lobula plate transfer information from the optic lobes into the midbrain and are involved in controlling optomotor responses. Optomotor responses are reflexive head and body movements that attempt to stabilize the retinal image and control body orientation during walking and flight. The lobula plate functions in a similar fashion to the nuclei of the AOS and pretectal NOT in mammals (see Section 3.2). Extensive studies on the neurons of the lobula plate show that these cells have response properties very similar to those expected from the final subtraction stages of the Reichardt and Energy models (Egelhaaf et al., 1989), as is the case in the AOS and NOT of mammals (Ibbotson et al., 1994; Ibbotson and Clifford, 2001a,b). That is, signals are already motion opponent. Elementary motion detector units in the fly can be excited by stimulation of just two receptor cells in adjacent facets of the eye (Kirschfeld, 1972; Franceschini et al., 1989). The discrete nature of local motion interactions in insects should assist in tracing movement signals at the electrophysiological and anatomical levels. Given the clear response properties in the lobula plate and the highly repetitive and organized nature of the medulla and lamina, it is interesting to search for the elements of the optic lobes that provide the input to the lobula plate neurons. This search has the very real possibility of identifying the biological building blocks of elementary motion detectors (DeVoe, 1980; DeVoe and Ockleford, 1976; Gilbert et al., 1991). Fig. 14. Schematic of connections in the insect optic lobes. All elements shown are thought to be involved in motion detection. PR, photoreceptors; L2 and L4, two types of lamina monopolar cells; Am, lamina amacrine cells; T1, basket T-cells; Tm1, type 1 transmedullary cells; T5, bushy T cells; C2, type 2 centrifugal neurons. As in the vertebrate retina (Figs. 11 and 12), the interconnectivity between channels is very extensive. Gilbert et al. (1991) suggest that non-linear interactions occur at lamina-to-medulla connections, e.g. perhaps L2 to Tm1. The T4 cells are not shown in the diagram because their role in motion processing (if any) is not well established. T4s have their dendrites close to the terminals of the intrinsic transmedullary cells (itm) in the inner medulla and terminate in the lobula plate.

17 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Movement-specific responses have not been identified from cells in the lamina (Mimura, 1974) but responses are highly direction-selective in the lobula plate. The elements responsible for direction-selectivity must, therefore, be located in the circuitry between the output synapses of the lamina cells and the input synapses of the lobula plate. In the following account, we will start in the lobula plate (the most directional area) and move outwards towards the lamina (the least directional area). The dendrites of the lobula plate neurons receive retinotopically organized synaptic inputs from T4 and T5 neurons (Strausfeld and Lee, 1991). The T4 neurons have their dendrites in the medulla while the dendrites of the T5 cells reside in the outer stratum of the lobula (Fig. 14). It has been suggested that the T4 and T5 cells are functionally similar to retinal ganglion cells in mammals (Douglass and Strausfeld, 2001). The terminals of the T4 and T5 cells in the lobula plate are located in four main levels that have distinct preferred motion directions. T5 cells generate motion opponent responses to moving patterns that are similar to those of the lobula plate neurons (Douglass and Strausfeld, 1995). The receptive fields of T5 cells are, however, far smaller than those of lobula plate neurons. In contrast, T4 cells are only weakly directional (Douglass and Strausfeld, 1996). The T5 cells must either receive input from cells that are already post-synaptic to the final subtraction stage of the EMDs or their input synapses would have to form the neural substrate for that final subtraction. T4 cells may represent a non-opponent stage in the motion detection mechanism. Recordings from unidentified cells in the medulla have shown that there are directional and non-directional movement-sensitive elements (McCann and Dill, 1969; Mimura, 1971; DeVoe and Ockleford, 1976; DeVoe, 1980). DeVoe (1980) recorded from cells in the medulla that responded to moving gratings with maintained non-directional depolarizations but often had directional oscillations or spikes superimposed on the depolarized signal. This type of response may arise from a cell that forms part of the building block of an elementary motion detector. DeVoe (1980) suggested that the characteristic response waveforms could be explained by multiplicative inputs from lamina and medulla cells to the movement detector units. However, no anatomical evidence was then available to confirm this pathway. A major indicator for this idea was that motion in one direction in some medulla neurons produced clear second harmonic response components while motion in the opposite direction produced either no second harmonics or low-amplitude second harmonic components. As outlined in Section 2.4.1, the existence of second harmonics in the responses to moving gratings suggests a second-order non-linear mechanism in the motion detector. This is an expectation predicted for the components of both the Reichardt and Energy models (Egelhaaf et al., 1989; Emerson et al., 1992). Gilbert et al. (1991) suggest that the second-order non-linearity may occur between the outputs of L2 laminar neurons and medulla cells. The extracellular electrophysiology certainly points towards the medulla or the interface between the lamina and medulla as a likely source for motion detector interactions. What do we know of the anatomy of those areas of the optic lobe? The main inputs to T4 and T5 cells appear to arise from the transmedullary cells Tm1, itm and, in certain species, from Tm1a, Tm1b and Tm9 (Fischbach and Dittrich, 1989; Douglass and Strausfeld, 1998). It is thought that the terminals of the itm cells are presynaptic to the dendrites of T4 cells and that the terminals of Tm1, Tm1a, Tm1b and Tm9 cells are presynaptic to the dendrites of T5 cells (Fig. 14). Physiological evidence shows that some Tm cells are directional but that the responses are not fully motion opponent (Gilbert et al., 1991; Douglass and Strausfeld, 1995). It is probable that the Tm1 cells are one of the components that make up the elementary motion detectors proposed in the Reichardt and/or Energy models. It is of course essential that some type of lateral spatial interaction occur in the motion detectors between neighboring regions of the visual field. There are several possible sources of lateral interactions. Firstly, it appears that the lateral interactions might occur at the transition from the lamina to the medulla. The lamina contains several types of L-monopolar cells in each optic cartridge, the latter containing all the neural tissue that lies underneath each facet lens of the eye. L2 monopolar cells may provide inputs from adjacent optic cartridges to Tm1 neurons via lateral connections involving T1 basket cells and lamina amacrine cells (Fig. 14). T1 cells terminate in the medulla between the terminals of L2 monopolar cells and the dendrites of Tm1 cells and receive input from lamina amacrine cells, which receive their signals from several photoreceptors in different cartridges (Douglass and Strausfeld, 2001). Second, in the lamina, the L4 monopolar cells and the lamina amacrine cells (Fig. 14) provide a system of connections between retinotopic columns (Strausfeld and Campos-Ortega, 1973). In this case, the L4 neurons receive input from lamina amacrine cells and then distribute this information to L2 and Tm1 neurons in different columns. Finally, the C2 neuron provides feedback from the inner layer of the medulla back to the outer medulla and lamina (Strausfeld, 1976). The C2 neurons cross between retinotopic columns, thus providing another possible source for lateral interactions in the motion processing pathway (Fig. 14). Visually responsive neurons that send their axons from the midbrain area back, centrifugally, into the medulla have been identified in several insects (e.g. moths: Collett, 1970, 1971; Milde, 1993; butterflies: Ibbotson et al., 1991). In both moths and butterflies the dendrites of the centrifugal neurons are in the midbrain area occupied by the outputs of direction-selective neurons from the lobula plate. The large centrifugal neurons are highly direction-selective, so fully motion-opponent signals from the midbrain are sent into the distal layers of the medulla. This is of significance for any physiologist recording from small centrally directed neurons in the medulla because any observed directionality may be

18 426 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) the result of signals fed back from the midbrain rather than signals arising from elementary motion detectors. In conclusion, the insect optic lobes provide a neural substrate that has the potential to reveal the exact structure of the local motion detector networks in a biological system. Far more attention has been given to the large direction-selective output neurons of the optic lobes than to the complex neural networks that probably form their input. It is certainly technically difficult to record from the small neurons in the medulla but the rewards of doing so could be substantial. It would be greatly beneficial to follow the course taken by the pioneers that have attempted to record from the motion detector pathways in the insect medulla (e.g. DeVoe and Ockleford, 1976; DeVoe, 1980; Osorio, 1986; Douglass and Strausfeld, 1995, 1996, 2001). However, in trying to find the elementary motion detectors in the outer optic lobes we must be careful to take into account any feedback systems that provide fully motion opponent signals to the outer optic lobes from the midbrain, as found in butterflies (Ibbotson et al., 1991). 4. Adaptive mechanisms in motion detection 4.1. Perceptual consequences of motion adaptation Visual analysis of the world is an active process involving the continual adaptation of elementary processing units. Rapid neural adaptation is a fundamental property of vision with moment-to-moment relevance to our perception (Muller et al., 1999; Dragoi et al., 2002). Prolonged adaptation to a moving stimulus has profound perceptual consequences. When fixation is transferred to a stationary pattern, illusory motion is seen in the direction opposite to the adapting motion, but with little or no accompanying change in perceived position (Nishida and Johnston, 1999; Snowden, 1998). This motion aftereffect (MAE) has been known since ancient Greece, and has been studied extensively over the past 40 years (see Wade and Verstraten, 1998). Motion adaptation also affects the subsequent perception of moving stimuli, causing shifts in perceived direction (Levinson and Sekuler, 1976; Patterson and Becker, 1996; Schrater and Simoncelli, 1998; Rauber and Treue, 1999; Alais and Blake, 1999) and speed (Goldstein, 1957; Carlson, 1962; Rapoport, 1964; Thompson, 1981; Smith and Hammond, 1985; Muller and Greenlee, 1994; Clifford and Langley, 1996a; Bex et al., 1999; Clifford and Wenderoth, 1999; Hammett et al., 2000). In addition, the effect selectively impairs the ability to detect low contrast (Levinson and Sekuler, 1980) or incoherent motion (Raymond, 1993a,b; Hol and Treue, 2001). An early account of the neural basis of perceptual aftereffects was that adaptation satiates or fatigues cells sensitive to the adapting stimulus (Kohler and Wallach, 1944; Sutherland, 1961). When the period of adaptation ceases, the spontaneous discharge of the fatigued cells remains suppressed (Barlow and Hill, 1963). This produces a bias away from the adapted stimulus in the response of the population of cells sensitive to the adapted stimulus dimension, giving rise to a perceptual repulsion effect. So, for example, after adaptation to a downwards-moving pattern, a static pattern will appear to drift upwards due to fatiguing of cells preferring downwards motion (Wohlgemuth, 1911; Mather et al., 1998). Motion adaptation impairs the ability to detect subsequent motion in a direction-selective manner, such that motion coherence thresholds are maximally elevated in the adapting direction (Raymond, 1993a; Hol and Treue, 2001). These threshold elevations show complete inter-ocular transfer, demonstrating that they are of cortical origin (Raymond, 1993b). It is generally assumed that motion detection is determined by the responsiveness of the neuron most sensitive to the test direction. As a result of adaptation, the responsiveness of neurons tuned to the adapting direction is reduced most, with the reduction in responsiveness of any given neuron determined by the angle between the adapting stimulus direction and that neuron s preferred direction (Kohn et al., 2001) Function or fatigue? The importance of light adaptation by photoreceptor cells in the retina is well established, enabling our visual systems to operate in a vast range of conditions from near darkness to bright sunlight (Barlow, 1969; Laughlin, 1994). Adaptation at subsequent stages of the visual system is also well documented, but has often been viewed as a limitation of the system associated with neural fatigue (Kohler and Wallach, 1944; Sutherland, 1961). However, physiological data from area 17 of cat show that adaptive contrast gain control mechanisms operate at a cortical level (Ohzawa et al., 1982), and suggest that transient temporal mechanisms might adapt on the basis of stimulus motion or temporal modulation to improve temporal frequency discrimination (Maddess et al., 1988). Physiological studies on the pattern-specificity of the neuronal response to motion adaptation (Hammond et al., 1989; Saul and Cynader, 1989) and demonstrations of the storage of aftereffects (Wohlgemuth, 1911; Wiesenfelder and Blake, 1992) provide further evidence that there is more to motion adaptation than neural fatigue. If cortical adaptation cannot be attributed to neural fatigue, it is reasonable to ask whether it serves a function analogous to light adaptation in the retina. The retina codes variations in luminance by adapting to, and hence discounting, the mean luminance. Light adaptation has clear functional benefits in ecological terms, allowing the visual system to operate over a huge range of light levels. While information about the illuminant is discarded, this is of little relevance in comparison to the preservation of luminance changes carrying information about the structure of the environment. However, when one considers motion adaptation rather than light adaptation, the appropriateness of such a strategy is less clear.

19 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) It has been argued that, in flying insects, it is more important for motion-sensitive neurons involved in the stabilization of flight to optimize their sensitivity to changes in image motion rather than to provide an accurate measure of absolute speed (Shi and Horridge, 1991). The logic of this argument is that, to maintain stability in flight, it is more important to be able to detect small perturbations in trajectory rather than to be continually reminded of the speed of flight. It is important to realize that this argument only holds for the stabilization-system. For other behaviors, such as measuring the distance traveled during foraging, insects use information on the absolute speed of optic flow during forward flight (Esch et al., 2001; Srinivasan et al., 2000). It is probable that insects use both absolute speed information and changes in speed to obtain their full repertoire of actions. Different types of cell specialized for forward flight detection have been identified, some showing phasic response properties, presumably for detecting changes in speed (Ibbotson, 1991a, 1992), while others maintain a steady firing rate during stimulation (Ibbotson, 1991b). Recordings from the neurons that continue to fire throughout a period of stimulation have shown that the mean level of the response decreases, corresponding to a drop in absolute sensitivity to motion (Ibbotson and Goodman, 1990; Ibbotson, 1992; Maddess and Laughlin, 1985). At the same time, there is an increase in the magnitude of changes in response to variation in image motion around the adapting level (Fig. 15), corresponding to an improvement in differential motion sensitivity (Maddess and Laughlin, 1985; Maddess et al., 1991; Shi and Horridge, 1991). In mammalian vision, one can think of situations where sacrificing information about absolute motion for enhanced differential motion sensitivity would be advantageous; a bear fishing in a stream, for example, could use differences in the speed of motion to detect the presence of its prey. But there are also situations, as with insects, where accurate estimation of absolute motion appears important; e.g. in predicting the trajectories of moving objects and the guidance of pursuit eye movements (Priebe et al., 2001). Thus, while the benefits of enhanced sensitivity to changes in luminance and changes in motion might appear analogous, the cost of discarding information about absolute motion seems much higher than the cost of losing information about the mean light level. Intriguingly, a recent report by Fairhall et al. (2001) suggests that rapid adaptation of the input/output relationship of the fly H1 neuron to different distributions of stimulus motion need not necessarily mean that information about the adapting level be discarded. Instead, Fairhall et al. (2001) find that information about the statistics of the stimulus ensemble are encoded by the statistics of the interspike interval distribution on a timescale only slightly longer than that of rapid adaptation Informational basis of motion adaptation From an information-processing standpoint, a possible function of motion adaptation is to work towards the robust and efficient transmission of signals coding for image motion. The constraints on neural information transmission are that signals must be passed through channels of limited bandwidth that are subject to transmission errors (Attneave, 1954; Barlow, 1961; Laughlin, 1989; Clifford and Langley, 1996a). These limitations are analogous to those faced in telecommunications applications where it is often advantageous to code signals adaptively so that the best compromise can be reached between maximizing effective bandwidth and minimizing the effects of transmission errors. Consequently, functional ideas about adaptation have been motivated by two main considerations: self-calibration and dynamic range optimization. Self-calibration is the property of a system to change itself in response to changes in the environment (recalibration) and to adjust to perturbations within the system in an unchanging environment (error-correction) (Andrews, 1964; Rushton, 1965; Ullman and Schechtman, 1982). Dynamic range optimization tends to reduce redundancy in the responses of individual sensory neurons (Attneave, 1954; Barlow, 1961, 2001), maximizing the effective bandwidth available for the transmission of novel information about the stimulus (Srinivasan et al., 1982; Laughlin, 1989; Clifford and Langley, 1996a). Empirical support for these ideas comes from electrophysiological studies of the fly H1 neuron which have shown formally that adaptation to motion tends to maximize information Fig. 15. Neuronal model of the effect of adaptation on absolute and differential sensitivity to the speed of image motion. (A) Adaptation produces a rightward shift of the response function. (B) This rightwards shift causes the neuronal response to drop over time. (C) The rightward shift positions a steeper part of the response function at the speed of the adapting stimulus, thus increasing the amount by which the response changes for a given change in speed.

20 428 C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) transmission (Brenner et al., 2000; Fairhall et al., 2001). The principle of redundancy reduction can be extended from single neurons to populations of neurons by adaptively decorrelating (Barlow and Foldiak, 1989) or orthogonalizing (Kohonen and Oja, 1976) their responses, and may be applicable to the coding of motion in human visual cortex (Clifford et al., 2000; Clifford, 2002) Dynamics of motion adaptation Adaptation to motion has been shown to generate large, robust aftereffects with identified neural correlates in the cat (Hammond et al., 1988; Giaschi et al., 1993), monkey (Petersen et al., 1985; van Wezel and Britten, 2002) and human cortex (Tootell et al., 1995; He et al., 1998; Culham et al., 1999; Huk et al., 2001). The principal neural substrate of the MAE in human visual cortex is believed to be the human homologue of monkey area MT (Fig. 16; Tootell et al., 1995; He et al., 1998; Culham et al., 1999; Huk et al., 2001), in which the vast majority of neurons are strongly direction-selective (Albright et al., 1984). A behavioral analogue of the MAE has also been observed in the optomotor response of the blowfly (Srinivasan and Dvorak, 1979). Neural correlates of motion adaptation similar to those observed in the mammalian cortex have been observed in direction-selective neurons in a range of insects (fly: Maddess and Laughlin, 1985; bee: Ibbotson and Goodman, 1990; butterfly: Maddess et al., 1991). Human psychophysical studies have shown that perceived speed is affected by prior adaptation to motion (Thompson, 1981; Smith, 1987), and even to stationary stimuli (Held and White, 1959; Clifford and Wenderoth, 1999). When adapting and test stimuli have the same contrast, speed and direction, perceived speed is consistently decreased by adaptation (Carlson, 1962; Rapoport, 1964; Thompson, 1981; Muller and Greenlee, 1994). Correspondingly, the perceived speed of a constantly moving stimulus decreases as a function of adaptation duration (Goldstein, 1957), decaying exponentially to a steady-level (Clifford and Langley, 1996b; Bex et al., 1999; Hammett et al., 2000). As the perceived speed of a constantly moving stimulus decreases, sensitivity to modulations or increments in speed is enhanced (Clifford and Langley, 1996b; Bex et al., 1999; Clifford and Wenderoth, 1999), at least for luminance-defined motion (Kristjansson, 2001), suggesting that an accurate representation of absolute speed is sacrificed for greater differential sensitivity. Speed increment thresholds remain approximately proportional to perceived speed during adaptation and recovery from adaptation (Bex et al., 1999; Clifford and Wenderoth, 1999) so that, as perceived speed decreases through exposure, the ability to detect small changes around that speed improves. The psychophysics of human motion adaptation parallels closely electrophysiological data recorded from the direction-selective H1 neuron in the lobula plate of the fly (see Section 3.4) in both form and time course (Clifford and Langley, 1996b). In the fly, prolonged exposure to Fig. 16. The motion aftereffect and human area MT. (a) Stationary view of the stimulus used by Tootell et al. (1995). (b) Human cortical visual area MT (V5) activated by that stimulus. The brain is shown in both normal and inflated format. Sulcal cortex (concave) is dark magenta and gyral cortex (convex) is lighter magenta. The functional magnetic resonance imaging (fmri) activity produced by moving minus stationary rings is coded in a pseudocolor scale varying from saturated magenta (threshold) to white (maximum activity). The prominent white patch on the bottom right lateral surface is area MT (V5). MT (V5) showed a clear increase in magnetic resonance signal amplitude during viewing of stationary stimuli when they were preceded by an adaptation stimulus moving continuously in a single direction. Reprinted with permission from Nature (Tootell et al., 1995). Macmillan Magazines Limited ( 1995).

21 maintained motion causes the response of H1 to decay exponentially over time to a steady level. The time constant of the response decay is of the order of 2 3 s for stimuli moving at around 50 per second (Maddess and Laughlin, 1985). In human vision, reported time constants range from 1 to 16 s dependent upon stimulus parameters (Clifford and Langley, 1996b; Bex et al., 1999; Hammett et al., 2000) Directionality of motion adaptation C.W.G. Clifford, M.R. Ibbotson / Progress in Neurobiology 68 (2003) Clifford and Wenderoth (1999) found that adaptation to motion per se is not required to enhance differential speed sensitivity in humans. Adaptation to temporal modulation in the absence of net motion was found to produce significant improvements in discrimination around the subjective matching speed. Discrimination thresholds were found to decrease in proportion to perceived speed, regardless of the direction of motion or orientation of the flickering grating. Thus, it appears that, in human vision, enhancements in differential speed sensitivity are driven largely by adaptation to temporal modulation rather than to motion itself. Curiously, the finding that motion adaptation is driven by temporal modulation rather than motion per se is analogous to electrophysiological data from the H1 and HSE neurons of the fly lobula plate (Borst and Egelhaaf, 1987) but distinct from those for motion-sensitive neurons in wallaby NOT (Clifford et al., 1997; Ibbotson et al., 1998). Borst and Egelhaaf (1987) measured the time constant of the decay of the response to impulsive (two-frame) motion in four conditions: control (no adaptation); preferred motion adaptation; anti-preferred motion adaptation; and adaptation to counter-phase flicker. In all except the control condition they found that the decay time constant reduced to between 20 and 40% of its unadapted value, suggesting that adaptation in that system, as for human speed perception, is driven by temporal modulation (Fig. 17). In the NOT of the mammalian wallaby, preferred direction motion causes the most significant adaptation. Anti-preferred motion or flicker induces some adaptation, but it is far weaker than that induced by preferred direction motion (Clifford et al., 1997; Ibbotson et al., 1998). However, the use of the time constant of the decay of the response to impulsive (two-frame) motion as an index of adaptation has recently been criticized on the basis that changes in the time constant as a function of adaptation can be modeled by the introduction of fixed high-pass temporal pre-filters prior to motion computation (Harris and O Carroll, 2002). Thus, a caveat must be applied to the use of impulse response data to infer the determinants of motion adaptation as in the wallaby experiments. Indeed, Harris et al. (2000) have shown that adaptation to motion in the HS neuron of the fly lobula plate is not simply driven by the associated temporal modulation of contrast. This work is discussed in detail in Section 4.6. In humans, prolonged exposure to a moving pattern affects not only the perceived speed but also the perceived Fig. 17. Motion-dependence and direction-selectivity of adaptation. Normalized decay time constants of the impulse responses of the insect H1 neuron (black) and DS neurons in the NOT (gray) compared with human psychophysical perceived speed data (hashed). The response of the insect H1 neuron is adapted to a similar degree by preferred direction motion, anti-preferred motion and flicker (Borst and Egelhaaf, 1987). The human perceived speed data follows a similar pattern (Clifford and Wenderoth, 1999). In contrast, the response of the NOT neurons is much more strongly adapted by preferred direction motion than by anti-preferred motion or flicker (Clifford et al., 1997; Ibbotson et al., 1998). direction of subsequent motion (Levinson and Sekuler, 1976; Patterson and Becker, 1996; Schrater and Simoncelli, 1998; Rauber and Treue, 1999; Alais and Blake, 1999). This phenomenon, known as the direction aftereffect (DAE), differs from the classical MAE in that a moving test stimulus is used to measure the DAE. For angles up to around 100 between the directions of motion of the adapting and test patterns, the perceived direction of the test pattern tends to be repelled away from the adapting direction. The magnitude of this repulsion can be as much as 40 for adapter-test angles around 30. For larger obtuse angles between adapting and test directions, the perceived direction of the test tends to be attracted towards that of the adapter (Schrater and Simoncelli, 1998). The magnitude of this attraction effect is smaller than that of the repulsion, peaking at around 15 for angles of around between adapter and test directions. The effect of motion adaptation on subsequent direction discrimination depends upon the angle between the adapting direction of motion and the baseline test direction around which discriminations are made. For parallel adapting and test directions, Phinney et al. (1997) found reductions in direction discrimination thresholds of around 20% while Hol and Treue (2001) found little or no effect of adaptation. This discrepancy in the magnitude of the effect of adaptation on subsequent discrimination around the adapting direction remains mysterious. However, it is interesting to note that Hol and Treue (2001) also report smaller effects of adaptation on motion detection than did Raymond (1993a), suggesting that their adaptation paradigm might somehow be less powerful