A relationship between behavioral choice and the visual responses of neurons in macaque MT

Transcription

1 Visual Neurscience (1996), 13, Printed in the USA. Cpyright 1996 Cambridge University Press /96 $ A relatinship between behaviral chice and the visual respnses f neurns in macaque MT K.H. BRITTEN, 1 W.T. NEWSOME, 1 M.N. SHADLEN, 1 S. CELEBRINI, 1 AND J.A. MOVSHON 2 1 Department f Neurbilgy, Stanfrd University Schl f Medicine, Stanfrd 2 Hward Hughes Medical Institute and Center fr Neural Science, New Yrk University, New Yrk (RECEIVED February 24, 1995; ACCEPTED May 30, 1995) Abstract We have previusly dcumented the exquisite mtin sensitivity f neurns in extrastriate area MT by studying the relatinship between their respnses and the directin and strength f visual mtin signals delivered t their receptive fields. These results suggested that MT neurns might prvide the signals supprting behaviral chice in visual discriminatin tasks. T apprach this questin frm anther directin, we have nw studied the relatinship between the discharge f MT neurns and behaviral chice, independently f the effects f visual stimulatin. We fund that trial-t-trial variability in neurnal signals was crrelated with the chices the mnkey made. Therefre, when a directinally selective neurn in area MT fires mre vigrusly, the mnkey is mre likely t make a decisin in favr f the preferred directin f the cell. The magnitude f the relatinship was mdest, n average, but was highly significant acrss a sample f 299 cells frm fur mnkeys. The relatinship was present fr all stimuli (including thse withut a net mtin signal), and fr all but the weakest respnses. The relatinship was reduced r eliminated when the demands f the task were changed s that the directinal signal carried by the cell was less infrmative. The relatinship was evident within 50 ms f respnse nset, and persisted thrughut the stimulus presentatin. On average, neurns that were mre sensitive t weak mtin signals had a strnger relatinship t behavir than thse that were less sensitive. These bservatins are cnsistent with the idea that neurnal signals in MT are used by the mnkey t determine the directin f stimulus mtin. The mdest relatinship between behaviral chice and the discharge f any ne neurn, and the prevalence f the relatinship acrss the ppulatin, make it likely that signals frm many neurns are pled t frm the data n which behaviral chices are based. Keywrds: Visual crtex, Extrastriate, Middle tempral, Mtin sensitivity, Psychphysics, Discriminatin, Behavir, Mnkey Intrductin One f the basic prblems f neurscience is t discver the links between the activity f the elements f the nervus system and the behavir f rganisms. Over the last few years, we have explred the link between visual crtical activity and perceptual judgment by studying the respnses f neurns in area MT (r V5), a regin f the macaque extrastriate visual crtex, while Reprint requests t: William T. Newsme, Department f Neurbilgy, Stanfrd University Schl f Medicine, Stanfrd, CA 94305, USA. Present address f K.H. Britten: Center fr Neurscience, University f Califrnia, Davis, Davis, CA 95616, USA. Present address f M.N. Shadlen: Department f Physilgy, and Washingtn Reginal Primate Research Center, University f Washingtn, Seattle, WA 98195, USA. Present address f S. Celebrini: Centre de Recherche Cerveau et Cgnitin, CNRS-UPS, Faculte de Medecine Rangueil, Tuluse, France. mnkeys perfrmed a behaviral discriminatin task. Our results suggest that MT neurns may prvide the signals upn which the behaviral discriminatins are based (Britten et al., 1992), and are in gd agreement with evidence frm a variety f surces that suggests a critical rle fr MT in visual mtin prcessing (Zeki, 1974; Maunsell & Van Essen, 1983; Albright, 1984; Mvshn et al., 1985; Newsme & Pare, 1988; Salzman et al., 1992; Schiller, 1993). Our analysis f this questin has cncentrated n cmparing averaged neurnal respnses and behaviral judgments t the same visual stimuli, and using this cmparisn t deduce functinal relatinships. In this paper, we turn t the questin f hw n a particular trial neurnal and behaviral respnse are related t each ther, in an effrt t understand hw sensry signals infrm specific perceptual decisins. In these experiments, we simultaneusly measured the respnses f single MT neurns and behaviral judgments under cnditins in which the stimuli were nearly ptimal fr the discriminative capabilities f the neurns under study. Bth mea- 87

2 K.H. Briilen et al. surements are variable: under cnstant stimulus cnditins the mnkeys' chices will vary frm trial t trial, as will the respnses f MT neurns. If these neurnal signals frm the basis fr the perceptual judgments, then neurnal variability shuld be reflected in behaviral variability. In ther wrds, ver a set f identical trials, the activity f the neurn and the mnkey shuld be crrelated. Our analysis reveals that, n average, there is such a crrelatin: the mnkeys were mre likely t chse the directin preferred by an individual neurn when it fired mre strngly. This relatinship was nt due t variability in the visual stimulus itself, and was nly present when the mnkey made behaviral chices between visual stimuli whse directins f mvement differentially activated the neurn being recrded. Our bservatins supprt the idea that this psitive crrelatin is a cnsequence f the fact that the neurnal signals we recrded cntribute directly t the mnkey's chice behavir. Preliminary accunts f these results have appeared elsewhere (Britten et al., 1988; Newsme et al., 1989), as has a reprt f similar bservatins in area MST (Celebrini & Newsme, 1994). we were able t measure the sensitivity f bth single cells and f mnkeys t the same weak, spatially distributed mtin signal. Task and training We measured psychphysical perfrmance in a tw-alternative frced-chice prcedure in which eye mvements were measured bth t ensure crrect fixatin and t prvide the perant respnse which indicated the chice n a trial. This task is schematically illustrated in Fig. 1. On each trial, the mnkey was first required t maintain fixatin n a small spt f light, and the trial was terminated if the mnkey brke fixatin at any time. Once the mnkey had maintained fixatin fr 300 ms, the mtin stimulus was turned n within an aperture centered n the receptive field, and remained n fr 2 s. The fixatin pint and stimulus were then turned ff and tw saccade targets were illuminated, crrespnding t the tw pssible directins f mtin, always 180 deg apart. The mnkey indicated its chice by making a saccade t ne f the tw targets. A crrect chice Methds The results reprted in this paper are based n a new analysis f data frm experiments that we have fr the mst part described previusly. The experimental methds are utlined here, and we refer the reader t ur earlier reprts fr mre detailed descriptins (Newsme & Pare, 1988; Britten etal., 1992, 1993). receptive field "Preferred" target Subjects and surgery These experiments were perfrmed n fur adult rhesus macaques (Macaca mulatt), three male and ne female. Three f these mnkeys were subjects in previusly published physilgical experiments (mnkeys E, J, and W); the furth (mnkey K) was added later. Each mnkey was implanted with a stainless-steel head hlder and scleral search cil (Judge et al., 1980), and trained t criterin n a directin discriminatin task (see belw). A stainless-steel recrding cylinder was then implanted ver ccipital crtex, and recrding experiments were initiated. All animal care and surgical prcedures cnfrmed with NIH guidelines fr the care and use f labratry animals. Visual stimuli The stimuli in these experiments were fields f dynamic randmly psitined dts rapidly pltted (6.67 khz) n the face f a CRT screen (HP 1321 r Xytrn A21, P4 phsphr). The stimulus cvered a square 20 deg n a side, but was usually masked dwn t a smaller area matched t the receptive field f the neurn under study. Each dt was deg in diameter and f very high cntrast; the average luminance was 0.6 cd/m 2. A specifiable fractin f the dts carried a unidirectinal apparent mtin signal whse speed was ptimal fr the neurn under study. We varied the strength f the mtin by specifying the prbability with which any particular dt wuld be pltted in apparent mtin. The ther dts were psitined randmly t create masking mtin nise. We give the strength f the stimulus as the prprtin f cherently mving dts, here termed the cherence f the visual stimulus (in previus reprts we have used the term crrelatin fr this value). By varying cherence, B fixatin pint stimulus targets Fixatin pint 2 secnds "Null" target * chice Fig. I. Schematic f the behaviral task and stimulus cnfiguratin used in these experiments. A: Spatial cnfiguratin. The uter circle dentes the receptive field f the neurn under study, and the inner shaded regin the randm dt stimulus. This stimulus can be in either the neurn's preferred directin r its ppsite ("null"). Respnse targets, which were small spts f light frm prjectin LEDs, were aligned with the tw pssible directins f mtin alng a diameter f the stimulus. In sme circumstances, we wuld shift these away frm cllinear alignment t reduce chice biases. B: Timing f events in a trial.

3 Behaviral chice and MT respnses 89 was rewarded with a drp f water r juice, and an incrrect chice was punished with a brief time-ut perid. Trials were presented in blcks in which the tw directins were equiprbable and randmly interleaved, and in which the mtin strength was cntrlled by the methd f cnstant stimuli. bth "preferred" and "null" directin chices in respnse t any particular stimulus. This allwed us t cnsider whether a relatinship existed between behaviral chice and neurnal respnse even under cnditins when the stimulus did nt vary frm trial t trial. T illustrate this relatinship, Fig. 2 shws Physilgical recrding During each recrding sessin, we intrduced single-unit electrdes (10-30 nm tip expsure; Haer, Brunswick, ME, Micr Prbe, Clarksburg, MD) int MT until a suitable single unit was islated. We mapped the receptive field using cnventinal gemetric targets, chse a screen aperture s that the stimulus just filled the receptive field, and established the preferred speed and directin using 100% cherence randm dt stimuli. We then presented blcks f trials cntaining stimuli f several cherence values spanning threshld, mving either in the preferred directin r its ppsite (the "null" directin). Typically, 15 trials per cnditin were presented in each blck, and we cntinued t present blcks f trials as lng as islatin culd be maintained and the mnkey cntinued t perfrm the task I % cherence um null decisi preferred decisins i i i i Neurnal data base We accepted neurns fr analysis if they culd be held lng enugh t cmplete an average f at least seven trials per cnditin, and if their respnses were directin selective. Specifically, we required that the smallest respnse t the preferred directin be larger than the largest respnse t the null directin at the highest cherence tested. This restricted ur sample t reliably directinally selective cells, mre than 90% f the cells we encuntered in MT. Of the 299 neurns described in this paper, 216 were included in previus reprts (Britten et al., 1992, 1993). The new data cme frm 38 cells frm mnkey K and 45 new cells frm mnkey E. ia i i 0% cherence Histlgy Mnkeys E and K are still participating in related experiments; we have histlgically cnfirmed the lcatin f the recrding sites in mnkeys J and W. Each animal was killed with an verdse f barbiturate and perfused with 0.9% nrmal saline fllwed by 10% frmalin fixative. The brain was remved, blcked, and allwed t sink in 30% sucrse. Frzen sectins f 48-fitn thickness were cut in a parasagittal plane, and alternate sectins thrugh the area f interest were stained with cresyl vilet r myelin by a mdified Gallyas methd (Gallyas, 1979). The area frm which we recrded was always clearly visible frm extensive guide tube damage psterir t MT. This regin always crrespnded well with a heavily myelinated regin n the psterir bank f the superir tempral sulcus, the best-knwn anatmical landmark fr MT (Allman & Kaas, 1971; Ungerleider & Mishkin, 1979; Van Essen et al., 1981). Results A measure f the assciatin between neurnal respnse and behaviral judgment T examine the relatinship between neurnal respnse and behaviral chice, we tk advantage f the fact that when ur mnkeys were presented with stimuli near threshld, they made % cherence Respnse (impulses/sec) Fig. 2. Respnse distributins frm a representative MT cell which shwed a substantial trial-t-trial cvariatin between firing rate and behavir. Each panel represents a single stimulus cherence and directin; the -6.4% cnditin crrespnds t null directin mtin. The upper histgrams (stippled) shw firing rates n trials n which the mnkey decided in favr f the preferred directin, and the lwer histgrams (pen) shw the firing rates n trials n which the mnkey made the ppsite decisin.

4 90 K. H. Britten et al. the respnses f an MT neurn whse discharge was related bth t the stimulus and t the mnkey's behaviral chice. Each pair f histgrams shws tw distributins f firing rates fr the respnses f this neurn t a single stimulus type. The pair in the upper panel shws respnses t mtin in the preferred directin at 6.4% cherence. The pair in the middle panel shws respnses t 0% cherence, and the pair in the lwer panel shws respnses t mtin in the null directin at 6.4% cherence. The relatinship between the visual stimulus and respnse can be seen in the trend fr average respnses t increase frm the bttm t the tp f the figure. The relatinship between the mnkey's chice and respnse can be seen in each panel: the upper, stippled histgram shws the distributin f respnses n trials n which the mnkey judged the mtin t be in the neurn's preferred directin, while the lwer, unfilled histgram shws the distributin f respnses n trials n which the mnkey judged the mtin t be in the neurn's null directin. Fr all three stimulus cnditins, the upper histgram is shifted t the right with respect t the lwer ne, shwing that the neurn fired mre strngly n trials n which the mnkey made decisins in favr f the neurn's preferred directin. This shift was significant fr each f the lwer tw pairs (/-test, P< 1). T avid distributinal assumptins inherent in the Mest, we elected t use a nnparametric statistic t examine the reliability f this effect. We devised a methd based n signal detectin thery, which is analgus t the receiver perating characteristic (ROC) analysis that we and thers have used previusly t describe neurnal respnses (Green & Swets, 1966; Barlw et al., 1971; Chn et al., 1971; Tlhurst et al., 1983; Bradley etal., 1987; Vgels & Orban, 1991; Britten et al., 1992). Fr each pair f distributins like thse shwn in Fig. 2, we calculated an perating characteristic in a manner identical t ROC analysis; the area under such a curve (nt shwn) captures the amunt f verlap f the distributins (Green & Swets, 1966; Bamber, 1975). The values fr the three pairs f distributins shwn in Fig. 2 are 0.57, 0.78, and The simplest way t cnceptualize these values is as the prbability that, given ne draw frm each distributin, the value frm the preferred-chice distributin wuld be larger. Suppse we were given data frm tw trials, n ne f which the mnkey made a "preferred" chice and the ther f which the mnkey made a "null" chice, but that we were nt tld which trial was which. Our measure gives the prbability that an bserver, given nly the firing rates fr tw trials, wuld be able t identify accurately which f the trials led t the mnkey making each chice. A value f 0.5 f curse represents chance perfrmance, and a value f 1.0 represents a perfect assciatin between neurnal and behaviral respnse, whse sense is such that the mnkey's chice crrespnds t the neurn's preferred directin. We term this value chice prbability, t emphasize its status as an indicatr f the accuracy with which neurnal respnse predicts the mnkey's chice. We calculated chice prbability fr all stimulus cnditins under which the mnkey made at least three crrect and three incrrect decisins. Fr each cell the chice prbability appeared t be independent f the stimulus cherence r f the magnitude f the neurn's respnse. We evaluated these impressins in tw ways. First, we averaged the chice prbability measurements fr each cherence level acrss all f the tested cells, and the results are shwn in Fig. 3A. Chice prbabilities were mdestly but reliably affected by stimulus directin (analysis f cvariance: F= 4.3, df= 2,2667, P = 14), but nt by the cherence f I s J 0.50 \ B Cherence (%) 1 10 Firing rate (impulses/trial) Fig. 3. Dependence f chice prbability n stimulus cherence and neurnal respnse. A: The average relatinship between stimulus cherence and chice prbability fr all 299 cells in ur sample. Negative cherence values indicate null directin mtin. Each pint represents the average chice prbability fr all cells fr which a chice prbability culd be measured at that cherence level. The errr bars represent the standard errr f the mean; large errrs typically mean thai the cherence level was rarely emplyed in ur experiments. B: The relatinship between mean neurnal firing rate and chice prbability fr 2668 stimulus cnditins fr the 299 cells in ur sample. Each pint represents the calculated chice prbability fr a single stimulus. The jagged line shws the running mean f 41 adjacent bservatins. the stimulus (F= 2.4, df= 1,2668, P = 4). This suggests that the strength f the stimulus, and by extensin the strength f the neurn's respnse, shuld nt have a cnsistent effect n chice prbability. Fig. 3B shws a direct examinatin f this questin, in which we plt the calculated chice prbability fr each f 2668 stimulus cnditins fr 299 neurns as a functin f respnse magnitude. The superimpsed jagged curve is a mving average f 41 adjacent bservatins. There was n strng relatinship between mean neurnal respnse and chice prbability, as lng as the firing rate was greater than 1-2 impulses/trial. We cannt be sure why the chice prbability cllapsed t 0.5 fr these very weak respnses, because the reliability f the measure is prbably pr when the spike cunts becme very lw. It is f interest t knw hw this measure relates t mre cnventinal measures f neurnal respnse such as changes in firing rate. Fig. 4 shws the relatinship between chice prbability and the rati f respnses measured fr trials n which the mnkey made crrect and incrrect behaviral judgments, fr the same 2668 stimulus cnditins and 299 neurns as in

5 Behaviral chice and MT respnses ,1 a 1.0i ^ fc Respnse rati ("preferred"/"null") Prprtin f trials Pig. 4. The relatinship between chice prbability and firing rate ratis between preferred and null chice trials, fr the same set f bservatins shwn in Fig. 3B. The marginal distributins shw the distributin f each measure, and the arrws indicate the crrespnding means. The jagged line is a running mean f 41 adjacent bservatins. Fig. 3B. The abscissa shws the rati f the firing rates measured fr the tw chice cnditins, and the rdinate shws the chice prbability fr the same stimulus cnditin. As in Figs. 3A and 3B, each stimulus cnditin was analyzed separately. The histgrams alng the tp and right margins shw the single-axis distributins. The chice prbability distributin shws a mean value acrss all bservatins near The respnse rati distributin reveals that the average firing rate change that prduced the bserved chice prbability values. On average, cells in MT fired abut 7% mre n preferred directin chice trials. The jagged curve shws the running mean f 41 adjacent bservatins. Over the central cre f the data distributins, this mean is apprximately linear. Tw sharp breaks in the average chice prbability ccur near respnse ratis f abut 0.7:1 and 1.5:1; utside this range, chice prbabilities shw n reliable dependence n respnse rati. These breaks ccur at chice prbability values f abut 0.3 and 0.75, suggesting that values f chice prbability mre extreme than these arise largely by chance. The analyses shwn in Figs. 3A and 3B suggest that it is legitimate t pl chice prbabilities acrss stimulus cherence levels. We scaled each bserved firing rate by the mean and standard deviatin fr that stimulus cnditin (z transfrm). This scaling has n effect n the rank rder f individual bservatins, and the chice prbability fr each cnditin is therefre unaffected. Hwever, it allwed us t cmbine spike cunts frm all cherence levels int a single pair f distributins, frm which we culd then calculate a single chice prbability fr each neurn. Using this methd, we derived chice prbabilities fr the preferred, null, and zer cherence stimulus cnditins fr each f the 299 MT cells. The distributins f these values are shwn in Fig. 5. We devised a permutatin test (Efrn & Tibshirani, 1993) t assess the significance f the deviatin f each bservatin frm chance (0.5). We randmly permuted the data fr each trial s that the assciatin f neurnal and behaviral respnse was ablished, while leaving the distributins f neurnal respnse and behaviral judgment untuched. We calculated the distributin f chice prbabilities expected in the absence f this assciatin frm 2000 such permutatins. We tk bserved chice prbability values that lay utside the central 95% f the distributin t be significant. Cells with significant values are shwn as stippled bars in the histgrams in Fig. 5. It is evident that relatively few cells had pled chice prbability values that achieved statistical significance when examined in islatin (85/296, 29%, fr the preferred directin, 70/287, 25%, fr zer cherence, and 74/293, 25%, fr the null directin). Of thse cases that did achieve significance, hwever, the great majrity (195/229, 84%) had chice prbabilities greater than 0.5, meaning that they had a significant psitive assciatin with the mnkey's behaviral chices. Of the 876 chice prbabilities in these three distributins, nly 36 (4%) had values significantly belw 0.5. We als extended ur j-transfrm methd t cmbine chice prbabilities acrss bth stimulus cnditins and neurns fr ur entire sample, t calculate a "grand" chice prbability reflecting the strength f the assciatin fr the entire pl f neurns fr each cnditin. We scaled the respnse fr each cnditin by its mean and standard deviatin, and then pled all f the resulting respnse distributins. We used the permutatin test described abve t evaluate the significance f the deviatin f each "grand" chice prbability frm chance. T cmpare tw "grand" chice prbabilities with each ther, we devised a secnd permutatin test. We frmulated the cmparisn as a test f the hypthesis that the bserved difference between tw chice prbabilities culd arise by chance, if bth sets f bservatins were sampled frm the same parent distributin. We tk the unin f the tw sets f z-transfrmed

6 92 K.H. Britten et al tw (P < 005). The values fr the preferred directin and zer cherence did nt differ frm ne anther (permutatin test, P = 0.399). Reliability f the chice prbability measure Chice prbability 1.00 Fig. 5. Distributins f neurnal chice prbability fr all 299 cells in the sample, cmpiled separately fr preferred directin (296 cases; a few lacked sufficient trials fr analysis), null directin (293 cases), and zer cherence (287 cases) respnses. Stippled bars indicate cases that were significantly different frm 0.5 by the permutatin test described in the text. T interpret the distributins shwn in Fig. 5, it is imprtant t have sme sense f the reliability f the individual bservatins frm which the distributins were frmed. We apprached the questin f reliability and repeatability empirically, by subdividing the data fr each cell int tw sets, using even-numbered trials fr ne set and dd-numbered trials fr the ther; we then separately calculated the chice prbability fr each subset. The results f this cmparisn are shwn in Fig. 6, in which the different estimates are pltted as a functin f the number f trials used fr the calculatins f chice prbability. Each vertical line cnnects the tw values f chice prbability btained frm the divided data set fr a single cell. The jagged curve shws the running mean f 21 adjacent bservatins. We draw several cnclusins frm this analysis. First, as wuld be expected, the cnsistency f the estimate f chice prbability imprved in an rderly way with the number f trials that cntributed t the measurement, as is evident frm the prevalence f shrt lines t the right f the plt and lng lines t the left. Secnd, the central tendency f the value f chice prbability was nt affected by the number f trials cntributing t the measurement, as is evident frm the lack f a cnsistent trend in the running mean. Third, mst values f chice prbability larger than abut arise largely by chance; they are nly seen near the left end f this graph, where the values are less reliable. This reinfrces the similar cnclusin we drew frm Fig. 4. Finally, the highly reliable pints near the right end f this plt reveal that individual cells can have chice prbabili- 1.0 respnses t frm a single jint distributin, retaining the assciatin between respnse and decisin. We randmly sampled this jint distributin t frm tw sets f values, each with the same number f bservatins as the test distributins, and calculated the difference in chice prbability between them. We estimated the distributin f such differences by repeating this 2000 times, and tk difference values that lay utside the central 95% f the distributin as significant. Fr the remainder f the paper, we will use statistics based n these permutatin techniques t test the significance f chice prbability values. These statistics are based in an unbiased way n the data themselves, and d nt embdy assumptins abut sample sizes and distributins that are implicit in mre cnventinal tests. In n case did the permutatin statistic lead t a cnclusin that cntradicted the utcme f a cnventinal distributinal test. The "grand" chice prbabilities calculated in this way fr the three distributins in Fig. 4 are 0.557, 0.553, and 0.534, respectively; these values are based n ver 37,000 trials fr the preferred and null stimulus directins, and ver 20,000 trials fr the zer-cherence trials. All three values were significantly different frm 0.5 {P < 005 in all cases), and the null directin value was significantly lwer than the ther 5 s Q. CD O Number f trials 300 Fig. 6. Repeated-measure reliability f the chice prbability fr 294 f the 299 neurns (five neurns were mitted because f an insufficient number f trials). Each vertical line cnnects tw values f the estimated chice prbability fr each neurn, calculated separately fr even and dd numbered trials. The slid line thrugh the center f the graph plts the running average f 21 adjacent bservatins, rdered by number f trials. Fur types f trial are used fr each calculatin (preferred and null directin decisins fr each subset f trials), and the values n the abscissa crrespnd t the average number f entries fr these trial types.

7 Behaviral chice and MT respnses 93 ties that differ bth frm ne anther and frm the ppulatin mean. A secnd way t visualize individual cells' data is shwn in Fig. 7, in which the tw repeated values f chice prbability are pltted against ne anther. Cells whse values were significant n even trials are pltted as plus signs, cells whse values were significant n dd trials are pltted as crsses, cells whse values were significant n bth subset analyses are pltted as stars (superimpsed plus signs and crsses), and cells whse values were significant n neither subset are pltted as small dts. The sensitivity f the permutatin statistic is revealed by the pint at which significant cells appear n either axis; naturally this sensitivity will depend n N. While there is substantial scatter f these data abut the diagnal, it is plain that there was a general tendency fr cells t have similar values in the tw subset analyses. The prevalence f dubly significant cells in the tp right quadrant (asterisks) is further evidence f the verall repeatability f the measure. Overall, the crrelatin between the tw values f chice prbability was 0.505, which was highly significant (F= 99.7, df= 1,292, P< 001). There was n difference n average between the chice prbabilities measured n even and dd trials (permutatin test, P = 60). The magnitude f the cvariance in this plt als reveals, like Fig. 6, that individual cells can have reliably different chice prbabilities. We will return later t the questin f whether the cells having higher chice prbabilities differ in ther ways frm their less well-assciated cmpanins. Stimulus effects The analysis in the preceding sectin reveals that fr ur ppulatin f neurns, chice prbability departed frm the value f 0.5 expected by chance. We cnclude that there is a systematic relatinship between the firing rates f MT neurns and the decisins made by the mnkey n single psychphysical tri Chice prbability (even trials) 1.0 Fig. 7. Cmparisn f chice prbability calculated frm the dd and even trial subsets used in Fig. 6. Values that differed significantly frm chance by the permutatin test n the even subset are shwn by +, n the dd subset by x, n bth subsets by *, and n neither by a pint. als. The mst interesting interpretatin f this bservatin is that these neurnal signals cntribute directly t the decisins made by the mnkey, and that variatin in the signals causes variatin in the mnkey's behavir. Such a causal relatinship shuld be influenced by changes in the visual stimulus, and in this sectin we explre the effects f several different stimulus manipulatins designed t clarify the relatinship between neurnal activity and behaviral chice. Trial-by-trial stimulus variatin We designed the first such manipulatin t explre the pssibility that the surce f ur effect was trial-by-trial variatin in the stimulus itself. Stimuli f a given directin and cherence nrmally differed frm trial t trial in the exact placement f dts in space and time, and cnsequently in the precise mtin signal presented. Sme trials by chance included mre preferred directin mtin, and n such trials ne wuld expect an increased respnse frm the neurn and als an increased prprtin f preferred directin judgments; ther trials included less preferred directin mtin, which shuld lead t the reverse utcme. This effect, were it significant, culd prduce the phenmenn that we are describing. T cntrl fr this pssibility, in sme experiments we presented stimuli withut this randm variatin. That is, fr each set f stimulus parameters, precisely identical dt patterns were presented n every trial. We asked whether the chice prbabilities were affected by the stimulus variatin. We perfrmed this cntrl in tw ways. First, we cllected data frm 43 cells in mnkeys J and W withut trial-t-trial stimulus variatin. We cmpared the magnitude f their chice prbabilities with thse f 108 cells frm the same tw mnkeys in which stimulus variatin was present. Fig. 8A shws the distributins f chice prbability fr these tw grups f cells. The upper panel shws the distributin f chice prbabilities measured with trial-t-trial stimulus variatin; the lwer panel shws the distributin measured with the same stimulus presented every trial. The tw sets f values are quite similar, and each deviates significantly frm chance (permutatin test, P < 005 in bth cases); the values are relatively lw because f differences between animals, which we cnsider belw. The values d nt differ significantly frm each ther (permutatin test, P = ), suggesting that remving stimulus variatin did nt discernibly affect the chice prbability. T prbe this issue mre directly, we cmpared the tw stimulus cnditins in the same neurns. We measured the chice prbabilities under bth cnditins in an additinal grup f 24 cells frm mnkey E. These measurements were made nly at 0% stimulus cherence, and trials f each type were randmly interleaved. Fig. 8B shws the results f this experiment, in the same frmat as Fig. 8A. There was n significant difference between the chice prbabilities under the tw cnditins (permutatin test, P = 0.68), and the mean f each distributin was significantly different frm 0.5 (permutatin test, P< 005). The tw measurements f the chice prbability fr each neurn were significantly crrelated (r = 0.452, F= 5.6, df= 1,22, P = 27). We cnclude that chice prbability was nt strngly influenced by stchastic variatin in the stimulus. This is cnsistent with ur earlier finding that this frm f stimulus variatin has n detectable effect n the trial-by-trial variance in cell discharge rate (Britten et al., 1993). Our interpretatin is simply that the effect f trial-by-trial stimulus variatin is negligible cmpared t ther factrs that cntribute t the cvariatin f neurnal respnse and behaviral judgment.

8 94 K. H. Britten et al. B 0.3 Stimulus variatin Stimulus variatin "55 u 5 c " 5 I" N variatin N variatin Chice prbability Chice prbability 0.8 Fig. 8. Dependence f chice prbability n the presence f trial-t-trial variance in the stimulus. A: Distributins f chice prbability fr 108 cells frm mnkeys W and J studied with stimulus variatin (upper histgram) cmpared with that fr 43 different cells frm the same mnkeys studied with n variatin (lwer histgram). B: Chice prbability distributins fr 24 cells frm mnkey E fr which we made interleaved measurements with bth types f stimulus. Bth measurements were made with 0% cherence stimuli which were randmly interleaved within blcks f trials that als cntained stimuli with nnzer cherence. Matching task demands t neurnal stimulus preferences Having established that chice prbability is nt determined by shrt-term stimulus fluctuatins, we nw turn t the relatinship between chice prbability and the "infrmativeness" f the neurn under study. Recall that in ur experiments, visual stimuli were carefully ptimized fr each neurn. Under these cnditins, an interpretatin f chice prbabilities greater than chance is that the neurn under study cntributes t the mnkey's judgments. Therefre, mdifying the stimulus cnditins t make the neurn's discharge uninfrmative fr a particular discriminatin shuld reduce the chice prbability t chance. We explred this using tw different stimulus manipulatins t render the cells' signals less directinal. In ne experiment, we remved the stimulus frm the receptive field f the neurn, but therwise left its parameters unchanged (i.e. we used the same size, speed, and directins f mvement). In a secnd experiment, we rtated the axis alng which the mnkey made his decisin while the stimulus remained lcated in the neurn's receptive field. T study the effect f stimulus placement, we altered the psitin f the stimulus relative t the receptive field by changing the lcatin f the fixatin pint, which we placed either s that the receptive field was centered n the stimulus, r s that its brder lay at least half a receptive-field diameter frm the nearest edge f the stimulus. The fixatin pint was placed s that the eccentricity f the stimulus was apprximately the same in each set f trials, and s that the target LEDs were rughly equidistant frm fixatin fr each cnditin. On- and ff-receptive field trials were interleaved. Fig. 9 illustrates the 5. I OnRF OffRF Chice prbability Fig. 9. Dependence f chice prbability n the lcatin f the stimulus. The chice prbability fr each f these 33 cells was measured in tw ways: with the visual stimulus either n (upper histgram) r ff (lwer histgram) the cells' receptive fields, as described in the text. All ther stimulus parameters remained the same.

9 Behaviral chice and MT respnses 95 effect f remving the stimulus frm the receptive field, fr 33 cells recrded frm mnkey E. The chice prbabilities fr the n-receptive field cnditin were significantly greater than chance (permutatin test, P< 005), and als systematically larger than thse fr the ff-receptive field cnditin (permutatin test, P < 005); the chice prbabilities fr the ffreceptive field cnditins were slightly but significantly lwer than 0.5 (that is, they were inversely related t the mnkey's decisin), a result that might be related t the mdulatin f MT neurn respnses by regins utside the cnventinal receptive field (Allman et al., 1985). We cnclude that the relatinship between firing rate and decisin was ablished (and perhaps reversed) by mving the stimulus ff the receptive field f the cell. The reductin f the chice prbability resulting frm remving the stimulus frm the receptive field f the cell might be a simple cnsequence f the reduced firing rates that this manipulatin prduced, and we wished t exclude this artifactual explanatin. Recall that in Fig. 3B, we examined the relatinship between chice prbability and neurnal firing rate. There was a weak relatinship between chice prbability and neurnal firing rate, evident fr very lw firing rates (<0.5 impulses/s). Amng the ff-receptive field data, such lw rates were bserved in three f 33 experiments, and remving these three cells did nt affect the utcme f the analysis. A mre subtle way t mdulate the infrmatin carried by MT neurns is t leave the stimulus centered n the receptive field, but t alter the axis f mvement s it was rughly rthgnal t the riginal, ptimal axis. This reduced the directinality f the cells, but because f their brad directinal tuning it did nt in general ablish visual respnses altgether. It als prved very difficult t eliminate directinality cmpletely, since the flanks f the directin tuning functin are ften asymmetric, and a few degrees f errr frm the perfect "balance pint" culd prduce substantial directinal respnses. Since in general this manipulatin did nt ablish visually evked activity, we were able t assign the "preferred" directin in these cases simply by chsing the directin eliciting the larger respnse. In this experiment the tw sets f trials were presented in blcks. We perfrmed this experiment n 45 cells frm mnkeys E and J, and the results are shwn in Fig. 10. The permutatin technique shws that the values f chice prbability fr bth stimulus cnditins were significantly greater than 0.5 (P < 005), but als that the chice prbability was significantly lwer fr the rthgnal cnditin (P < 005) than fr the n-axis cnditin. Thus, the relatinship between neurnal firing and behaviral chice was attenuated but nt ablished by this manipulatin, which attenuated but did nt ablish visually elicited respnses. Bth the ff-receptive field and ff-axis experiments allw the interpretatin that significant chice prbabilities were evident when the visual stimulus being discriminated was apprpriate t activate the neurn being recrded. If the neurn's signals were irrelevant t the behaviral judgment, then the value f chice prbability fell t values near chance. Time curse f the chice prbability signal In principle, the decisin-related activity we have described culd arise because the mnkey's decisins depended n the activity f the neurn being recrded, r because the neurn being recrded was influenced by the mnkey's decisin. We explred "5. I I Chice prbability Preferred axis Orthgnal axis [-, Pig. 10. Dependence f chice prbability n (he axis f mtin f the stimulus. The chice prbability fr each f these 45 cells was measured in tw ways: with the visual stimulus mving alng the neurn's preferred axis (upper histgram), r rughly rthgnal t it (lwer histgram). the directin f causality by examining the time curse f the firing rate differences that we bserved. We have until this pint described nly an integral measure, incrprating with equal weight all spikes recrded during the 2-s stimulus perid. T explre the dynamics f the relatinship, we examined hw firing rate changed with time within the stimulus perid. If the chice prbability were related t feedback frm the decisin that the mnkey made, we wuld expect the firing rate difference between "preferred-decisin" and "null-decisin" trials t be delayed until the perceptual judgment became mre reliable; in an average recrd the difference wuld evlve gradually during the stimulus perid. If, n the ther hand, it were the neurnal respnse that influenced the perceptual decisin, we wuld expect all spikes t be cunted mre r less equally, and the effect t emerge early and be mre r less statinary thrughut the respnse. T study this questin, we averaged the nrmalized respnse histgrams fr the 75 cells in the sample that shwed a significant psitive chice prbability (>0.5 nly). We averaged nly the respnses t 0% cherence stimuli, t avid cntaminatin by the dynamics f the directinal signal itself. Beginning 300 ms befre the nset f the stimulus, we made separate averaged rate recrds fr preferred decisin and null decisin trials, bth nrmalized t the peak f the preferred decisin histgram. The tw histgrams are superimpsed and pltted in Fig. 11 A, and the difference between them is pltted in Fig. 1 IB. This analysis reveals that the difference in firing rates emerged very early in the respnse-within apprximately 50 ms-and remained cnstant fr the duratin f the stimulus. We take this as evidence that significant chice prbabilities arise nt because f feedback frm the decisin itself, but because signals frm MT neurns are likely t feed frward int the deci- 0.8

10 96 K.H. Britten et al & 8 mnkey E Nl O Time (msec) mnkey K mnkey W mnkey J Time (msec) Fig. 11. Time curse f the firing rate difference that underlies the chice prbabilities. The 0% cherence respnses f 75 cells that shwed a significant chice prbability were cmbined. Each cell gave a pair f averaged respnse histgrams (bin width: 10 ms) crrespnding t preferred and null directin decisin trials. Each pair was nrmalized t the peak f the preferred directin histgram. A: Pled average respnse histgrams fr each respnse directin, with the upper representing the preferred directin decisin trials and the lwer shwing the null directin decisin trials. B: The difference between the tw respnses as a functin f time. Nte that the respnse difference is nly present during the visual stimulus perid, and nt during the fixatin perid prir t stimulus nset (arrws). sin prcess. Furthermre, the absence f a rate difference in the fixatin perid befre stimulus nset suggests that any effects f respnse bias r selective attentin which might be evident befre the stimulus are nt the surce f the chice prbabilities we bserved. Interanimal differences Our mnkeys differed frm each ther in their psychphysical perfrmance and in the sensitivity f their individual neurns (Britten et al., 1992). We were therefre interested t ntice that chice prbability als differed amng animals. Fig. 12 shws the distributins f the neurnal chice prbabilities individually fr each animal. The distributins are different fr each animal, and thse fr mnkey E and mnkey K are shifted t the right with respect t the ther tw. These tw animals each had relatively large numbers f significant chice prbability Chice prbability 1.00 Fig. 12. Chice prbability distributins frm the fur mnkeys (mnkey E: 97 cells; mnkey K: 38 cells; mnkey W: 87 cells; and mnkey J: 77 cells). The chice prbabilities were pled fr all stimulus cnditins. The stippled bars, as in Fig. 5, indicate neurns whse chice prbabilities differed significantly frm 0.5 n the permutatin test. values (mre than half the cells in each case), the great majrity f which were greater than 0.5. The mean f the distributin fr each mnkey was significantly greater than 0.5 (permutatin test, P < 005 fr mnkeys E, W, and K, P = 3 fr mnkey J). Althugh the value fr mnkey J was nly barely significant, substantially mre cells than expected shwed a significant chice prbability in this mnkey (24/77, 31%, stippled bars); tw-thirds f these values were greater than 0.5. It is wrth nting in this cntext that mnkey J differed frm the ther mnkeys in having higher psychphysical and neurnal threshlds (Britten et al., 1992). Cnsidering all animals, hwever, we did nt find any simple relatinship between neurnal chice prbabilities and either psychphysical r average neurnal sensitivity. Since the tw mnkeys with the largest average chice prbabilities (E and K) were als the last t be studied, we wndered whether the apparent differences between mnkeys actually reflected differences due t sme unknwn factr (such as changes in experimental prcedure) that ccurred with time. An analysis f cvariance revealed, hwever, that neurnal chice prbability varied significantly with mnkey (F = 6.6, df =

11 Behaviral chice and MT respnses 97 3,296; P = 004), but nt with the date f recrding (F= 9, df= 1,298, P = 0.66). Relatinship f chice prbability t neurnal threshld If we cnsider the chice prbabilities t reflect the cntributin f individual cells t the mnkey's psychphysical judgment, then sme cells might be mre strngly assciated with the decisin than thers. Naturally, we wndered whether the neurns with the highest sensitivities (which carried the "best" sensry signals) had higher chice prbabilities than thse with lwer sensitivity. Sensitivity f a neurn may be measured in many ways, but in previus wrk we have used a methd derived frm signal detectin thery t calculate a cherence threshld fr each cell, the cherence level at which the neurn achieved a criterin level f directinality (see figure legend and Britten et al., 1992 fr mre detail). The relatinship between chice prbability and neurnal threshld is shwn in Fig. 13. The crrelatin between these measures was 0.328, and was highly significant (r = , F= 35.5, df= 1,295, P< 001). The plt als shws the best linear relatinship derived frm a maximum-likelihd regressin analysis. Each mnkey's data are drawn with different symbls, and inspectin reveals that individual animals differed substantially frm ne anther. The tw mnkeys with the lwest average chice prbabilities (mnkeys J and W) are illustrated with pen symbls, and fr these mnkeys, the relatinship between chice prbability and threshld was nt significant (r = 74, F = 0.89, df= 1,162, P = 0.35). Fr the ther tw mnkeys, E and K, the relatinship was strng (r = , F = 60.6, clf= 1,131, p < 001). This relatinship suggests that neurns with lwer threshlds are mre clsely assciated with the mnkeys' decisins. Discussin We have previusly dcumented the exquisite mtin sensitivity f neurns in extrastriate area MT by studying the relatinship between their respnses and the directin and strength f visual mtin signals delivered t their receptive fields (Britten et al., 1992). These results suggested that MT neurns might prvide the signals supprting behaviral chice in visual mtin discriminatin tasks. T apprach this questin frm anther directin, we have nw shwn that signals carried by a single MT neurns are assciated, trial by trial, with the mnkeys' decisins. On a given trial, the mnkey was mre likely t make a decisin in favr f the preferred directin f a neurn when the neurn was firing mre vigrusly. The magnitude f this assciatin between neural respnse and behaviral chice was mdest, but given the large number f neurns in MT (as well as in ther areas), it is remarkable that such a crrespndence exists at all. Chice prbability as a measure We describe the relatinship we have uncvered with a measure we term chice prbability, a distributin-free metric clsely related t the ROC analysis f signal detectin thery (Green & Swets, 1966). We were, in fact, initially drawn t this apprach because f its cnceptual tie t signal detectin thery. In cntrast t traditinal uses f signal detectin thery, hwever, we d nt seek a relatinship between the stimulus and neurnal respnse; instead we use it t detect a relatinship between neurnal activity and the mnkey's chice. The measurement f chice prbability prved t be mre difficult than the ther kinds f neurnal respnse measurements that we have undertaken in this set f studies. As is graphically shwn in Fig. 6, chice prbability is a subtle quantity, nt well estimated frm small numbers f trials. The dependence f the reliability f the estimate f chice prbability n trial number is rughly as ne wuld expect, but the estimate des nt becme reliable until each chice distributin cntains rughly 100 trials. It is als imprtant t realize that fr these trials the stimuli must be near threshld s that the mnkey makes a useful number f errrs n the psychphysical task " S i mnkey E i mnkey J > mnkey K i mnkey W 10 Neurnal threshld cherence (%) 100 Fig. 13. Relatinship between chice prbability and neurnal threshld. The neurnal threshld was derived frm an analysis presented elsewhere (Britten et al., 1992); it crrespnds t the lwest stimulus cherence at which the neurn's respnse culd be used t distinguish preferred frm null directin mtin n 82% f trials. The relatinship prtrayed here differed cnsiderably acrss mnkeys; the tw animals with the highest average chice prbability (E and K) were the nly nes t shw a significant relatinship individually. These tw animals' data are drawn with slid symbls. Chice prbabilities were derived frm all stimulus cnditins cmbined; similar effects were bserved when chice prbabilities fr different directins f mtin were cmpared separately.

12 98 K.H. Britten et al. As a result, ur experiments barely sufficed t uncver the chice prbability effect that is the subject f this paper. Cnsider, fr example, the significance f an individual neurn's chice prbability. The average deviatin frm chance f the chice prbability effect is abut 5; this is smaller than the expected dispersin f the estimate fr the number f trials we btained fr mst cells. Therefre, tests f the significance f individual bservatins reveal significant effects nly fr abut 25% f cells (cf. Figs. 5 and 11); we rely upn cmbined measures fr ur strngest cnclusins. One defensible interpretatin f this is that individual neurns d nt in fact have reliable chice prbability values perhaps nly the ppulatin f neurns as a whle shuld be cnsidered t have a reliable assciatin with respnse. In fact, we believe that individual neurns d differ in their chice prbabilities. Fr example, neurns that have high sensitivity (lw threshlds) tend als t have high values f chice prbability (Fig. 13). Related bservatins The apprach f simultaneusly recrding neurnal activity and psychphysical chices near threshld is nt new, althugh it is rather rarely used. Lgthetis and Schall measured activity in MT simultaneusly with directinal decisins under uncertainty cnditins resulting frm bincular rivalry (Lgthetis & Schall, 1989), and sme f their cells shwed results qualitatively similar t urs. Curiusly, thugh, they fund apprximately equal numbers f cells with psitive and negative assciatin t the behaviral reprt. As nted abve, Celebrini and Newsme (Celebrini & Newsme, 1994) have used this methd in recrdings frm area MST. Related bservatins have als been made in the smatsensry system. Vallb and Jhannsn (Vallb & Jhannsn, 1976) reprted a similar effect in recrdings frm cutaneus mechanreceptr afferents. Dubner and his clleagues have recrded in bth the medullary drsal hrn and in SI while mnkeys were detecting temperature increments near threshld (Dubner et al., 1989; see als Muntcastle et al., 1990; Sinclair & Burtn, 1991 fr related bservatins in anther smatsensry cntext). These studies reprt chicerespnse assciatins qualitatively similar thse we have described, althugh the nature f smatsensry experiments makes it difficult t rule ut explanatins related t ther aspects f the animal's behavir. Fr example, the animal can change the way it tuches the discriminanda frm trial t trial, and thus the exact magnitude f the stimulus. In fact, Sinclair and Burtn cnclude that this was the principal cause f the crrelatin they bserved. Mechanism Plainly, we lean t the view that the chice prbability effect is a statistical signature f the cntributin that neurns in MT make t perceptual judgments. Befre we can assert this view with cnfidence, hwever, we must cnsider a variety f ther, less interesting, explanatins. The simplest pssible explanatin fr the relatinship we bserved is that it reflects the inherent trial-t-trial variability present in the stimulus itself. As a cntrl fr this, we presented stimuli which were identical fr any particular directin and cherence. In these cases, the chice prbability remained significant, and was as large as it was in the presence f stimulus variatin (Fig. 8). Our results are cnsistent with the idea that a cmpnent f the chice prbability culd be due t stimulus variatin, but it is nly a small part f the effect that can be explained in this way. Eye mvements are always a cncern in interpreting data frm experiments in alert mnkeys. In the present experiments, eye mvements were restricted by the cmputer (which abrted trials with excessive eye mvement), and mnitred nearly cntinuusly by the experimenter. It remains pssible that the small residual eye mvements culd mdestly influence the firing rates f the neurns we recrded. Hwever, fr eye mvements t prduce the chice prbabilities we bserve, these wuld nt nly have t mdulate neurnal discharge, but wuld have t d s in a manner that was crrelated with the animal's subsequent chice. The mst plausible pssibility is that the mnkey might within very narrw permitted limits attempt t track the seen mtin. But since such tracking wuld reduce retinal target speed and therefre reduce the neurnal respnse (Newsme et al., 1988), the effect wuld be t decrease rather than increase the measured chice prbability. Significant chice prbabilities culd als arise if linked changes in neurnal and behaviral sensitivity ccurred ver the curse f an experiment; just such an effect was indeed reprted by Zhary et al. (1994), wh shwed that during an experimental sessin, bth neurnal threshlds and behaviral perfrmance tended t imprve. The prtin f the chice prbability deviatin frm chance that can be due t this effect is, hwever, nt large (less than 05); this effect can als plainly nt be a factr fr stimuli f O 7 cherence. We perfrmed a smthing analysis t remve lng-term trends in the data that culd affect the calculatin f chice prbability; this analysis cnfirmed that the effect f these lng-term trends was negligible. Finally, we cnsidered the pssibility that glbal mdulatins f attentin r arusal state might jintly affect bth neurnal and behaviral sensitivity, artifactually creating a significant chice prbability. This culd result, fr example, if the mnkey made crrect chices mre ften n trials with heightened attentin and if the neurn were simply mre sensitive (i.e. mre directinal) n such trials (e.g. Mran & Desimne, 1985). Nte that this mechanism culd lead t a chice prbability fr any directinal neurn, whether r nt it cntributed t the mnkey's perceptual judgments. This culd nt, hwever, explain the existence f a chice prbability fr trials n which the mtin stimulus had 0% cherence. While neurnal activity might cnceivably be mdulated by attentin n such trials, there is n basis fr suppsing that the mnkey culd make mre "crrect chices" n these trials, since "crrect" chices are arbitrary at 0% cherence. On these trials, then, the mnkey's decisins wuld be randmly assciated with trial-t-trial fluctuatins in neurnal respnsiveness unless the bserved neurnal variability actually influenced the mnkey's chices; Fig. 5 shws instead that chice prbability was just as strng fr 0% cherence as it was fr ther stimuli. Mrever, we have presented evidence elsewhere that attentin has n discernible effect n the threshlds f MT neurns (Britten et al., 1992). The final evidence against such "glbal" explanatins cmes frm experiments perfrmed in the current study using ineffective r subptimal visual stimuli (Figs. 9 and 10). Recall that when the activity f a neurn was less relevant t the discriminatin being perfrmed, the chice prbability was reduced r eliminated.

13 Behaviral chice and MT respnses 99 Thus, under the cnditins we have tested, chice prbability is nt ubiquitus, but is mre rbust in the specific subset f neurns that prvides infrmatin relevant t the present task. Our interpretatin is that signals frm this subset f neurns are being used t frm the behaviral chice. Cnclusins We believe that finding a significant chice prbability fr a given pl f neurns prvides gd evidence fr the invlvement f the pl in perceptual judgments. We have in ther experimental and theretical wrk devted substantial effrt t trying t establish the size and nature f the neurn pls supprting perceptual chice (Britten et al., 1992; Shadlen et al., 1995). Data cncerning neurnal threshlds t visual stimulatin d nt by themselves prvide an adequate cnstraint fr such mdeling they are cnsistent with several very different ntins f hw "relevant" neurnal signals are selected and pled (Britten et al., 1992). Our chice prbability data prvide decisive additinal cnstraints. The first imprtant cnstraint is due t the fact that chice prbabilities were rughly equal fr stimuli mving in the preferred and null directins, as well as fr stimuli withut net mtin (Fig. 5). This suggests that signals frm relevant MT neurns are always mnitred when they can be infrmative, even when the stimulus mves ppsite t the neurn's preferred directin. Thus, we cnceive f perceptual decisins being derived frm the cmparisn f activity in separate pls f MT neurns preferring different directins, rather than frm particular neurns that are very active because the stimuli precisely match their preferences. The difference between the chice prbabilities in the tw directins (Fig. 3A) may prvide infrmatin abut the balance f these pls. A secnd cnstraint is due t the relatively lw values f chice prbability we bserved fr individual neurns. These values are nt cnsistent with mdels in which perceptual judgments are based n very small numbers f neurns, because these mdels wuld predict a much higher degree f assciatin between the relevant neurns and the behaviral judgments. We have therefre mdeled the way that pls f neurns culd cntribute t perceptual judgment, an exercise that we will describe in detail elsewhere (Shadlen et al., 1995). Our analysis prvides physilgists with a new way t prbe the cntributin f particular neurns t perceptin and actin. The traditins f sensry neurscience rely heavily n the indirect inference f a functinal rle fr a grup f neurns simply because they pssess a particular respnse prperty. Yet sensry signals in the brain serve many functins, and it is presumptuus t imagine that a particular neurn has a particular rle because we as physilgists tested its respnses in a particular way and fund it adequate. The measurement f chice prbability ffers a pwerfully different perspective n this crucial questin, and its presence allws us t make a principled argument fr the invlvement f neurnal activity in behavir. Acknwledgments We are grateful t Wyeth Bair, Laurence Malney, Brian Wandell, and Udi Zhary fr useful discussins and analytical help; we are als grateful t Judy Stein fr her expert technical assistance. We are grateful t Jhn Maunsell and an annymus referee fr helpful reviews. This wrk was supprted by grants frm the Natinal Eye Institute (EYO56O3 and EY020I7), and by the Hward Hughes Medical Institute. M.N. Shadlen was supprted by a Hward Hughes Medical Institute Pstdctral Research Fellwship fr Physicians, K.H. Britten was supprted by N1H training Grant NSO7158, and S. Celebrini was supprted by a pstdctral fellwship frm the CNRS, France. References ALBRIGHT, T.D. (1984). Directin and rientatin selectivity f neurns in visual area MT f the macaque. Jurnal f Neurphysilgy 52, ALLMAN, J.M.&KAAS, J.H. (1971). A representatin f the visual field in the caudal third f the middle tempral gyrus f the wl mnkey (Atus trivirgatus). Brain Research 31, ALLMAN, J.M., MEIZIN, F. & MCGUINNESS, E. (1985). Directin and velcity-specific respnses frm beynd the classical receptive field in the middle tempral visual area (MT). Perceptin 14, BAMBER, D. (1975). The area abve the rdinal dminance graph and the area belw the receiver perating characteristic graph. Jurnal f Mathematical Psychlgy 12, BARLOW, H.B., LEVICK, W.R. & YOON, M. (1971). Respnses t single quanta f light in retinal ganglin cells f the cat. Visin Research (Suppl.) 3, BRADLEY, A., SKOTTUN, B.C., OHZAWA, I., SCLAK, G. & FREEMAN, R.D. (1987). Visual rientatin and spatial frequency discriminatin: A cmparisn f single cells and behavir. Jurnal f Neurphysilgy 57, BRITTEN, K.H., NEWSOME, W.T. & MOVSHON, J.A. (1988). Assciatin between crtical unit activity and psychphysical respnse in alert mnkeys. Sciety fr Neurscience Abstracts 14, 458. BRITTEN, K.H., SHADLEN, M.N., NEWSOME, W.T. & MOVSHON, J.A. (1992). The analysis f visual mtin: A cmparisn f neurnal and psychphysical perfrmance. The Jurnal f Neurscience 12, BRITTEN, K.H., SHADLEN, M.N., NEWSOME, W.T. & MOVSHON, J.A. (1993). Respnses f neurns in macaque MT t stchastic mtin signals. Visual Neurscience 10, CELEBRINI, S. & NEWSOME, W.T. (1994). Neurnal and psychphysical sensitivity t mtin signals in extrastriate area MST f the macaque mnkey. Jurnal f Neurscience 14, COHN.T.E., GREEN, D.G.& TANNER, W.P.(1971). Receiver perating characteristic analysis. Applicatin t the study f quantum fluctuatin in ptic nerve f Rana pipiens. Jurnal f General Physilgy 66, DUBNER, R., KENSHALO, D.R., MAIXNER, W., BUSHNELL, M.C. & OLIVERAS, J.L. (1989). The crrelatin f mnkey medullary drsal hrn neurnal activity and the perceived intensity f nxius heat stimuli. Jurnal f Neurphysilgy 62, EFRON, B. & TIBSHIRANI, R.J. (1993). An Intrductin t the Btstrap. New Yrk: Chapman and Hall. GALLYAS, F. (1979). Silver staining f myelin by means f physical develpment. Neurlgical Research 1, GREEN, D.M.&SWETS, J.A. (1966). Signal Detectin Thery and Psychphysics. New Yrk: Jhn Wiley and Sns, Inc. JUDGE, S.J., RICHMOND, B.J. & CHU, F.C. (1980). Implantatin f magnetic search cils fr measurement f eye psitin: An imprved methd. Visin Research 20, LOGOTHETIS, N.K. & SCHALL, J.D. (1989). Neurnal crrelates f subjective visual perceptin. Science 245, 761. MAUNSELL, J.H.R. & VAN ESSEN, D.C. (1983). Functinal prperties f neurns in the middle tempral visual area (MT) f the macaque mnkey: I. Selectivity fr stimulus directin, speed and rientatin. Jurnal f Neurphysilgy 49, MORAN, J. & DESIMONE, R. (1985). Selective attentin gates visual prcessing in the extrastriate crtex. Science 229, MOUNTCASTLE, V.B., STEINMETZ, M.A. & ROMO, R. (1990). Frequency discriminatin in the sense f flutter: Psychphysical measurements crrelated with pstcentral events in behaving mnkeys. Jurnal f Neurscience 10, MOVSHON, J.A., ADELSON, E.H., GIZZI, M.S. & NEWSOME, W.T. (1985). The analysis f mving visual patterns. In Pattern Recgnitin Mechanisms, ed. CHAGAS, C, GATTASS, R. & GROSS, C, pp New Yrk: Springer-Verlag.

14 100 K.H. Britten et al. NEWSOME, W.T., BRJTTEN, K.H., MOVSHON, J.A. & SHADLEN, M. (1989). Single neurns and the perceptin f visual mtin. In Neural Mechanisms f Visual Perceptin. Prceedings f the Retina Research Fundatin, ed. LAM, D.M.-K. & GILBERT, CD., pp The Wdlands, Texas: Prtfli Publishing Cmpany. NEWSOME, W.T. & PARE, E.B. (1988). A selective impairment f mtin perceptin fllwing lesins f the middle tempral visual area (MT). Jurnal f Neurscience 8, NEWSOME, W.T., WURTZ, R.H. & KOMATSU, H. (1988). Relatin f crtical areas MT and MST t pursuit eye mvements. 11. Differentiatin f retinal frm extraretinal inputs. Jurnal f Neurphysilgy 60, SALZMAN, CD., MURASUOI, CM., BRITTEN, K.H. & NEWSOME, W.T. (1992). Micrstimulatin in visual area MT: Effects n directin discriminatin perfrmance. Jurnal f Neurscience 12, SCHILLER, P. (1993). The effects f V4 and middle tempral (MT) area lesins n visual perfrmance in the rhesus mnkey. Visual Neurscience 10, SHADLEN, M.N., BRITTEN, K.H., NEWSOME, W.T. & MOVSHON, J.A. (1996). A cmputatinal analysis f the relatinship between neurnal and behaviral respnses t visual mtin. Jurnal f Neurscience (accepted pending revisins). SINCLAIR, R.J. & BURTON, H. (1991). Tactile discriminatin f gratings: Psychphysical and neural crrelates in human and mnkey. Smatsensry and Mtr Research 8, TOLHURST, D.J., MOVSHON, J.A. & DEAN, A.F. (1983). The statistical reliability f signals in single neurns in cat and mnkey visual crtex. Visin Research 23, UNGERLEIDER, L.G. & MISHKIN, M. (1979). The striate prjectin in the superir tempral sulcus f Macaca mulatta: Lcatin and tpgraphic rganizatin. Jurnal f Cmparative Neurlgy 188, VALLBO, A.B. & JOHANNSON, R.S. (1976). Skin mechanreceptrs in the human hand: Neural and psychphysical threshlds. In Active Tuch: The Mechanisms f Recgnitin f Objects by Manipulatin, ed. GORDON, G., pp New Yrk: Oxfrd. VAN ESSEN, D.C., MAUNSELL, J.H.R. & BIXBY, J.L. (1981). The middle tempral visual area in the macaque: Myelarchitecture, cnnectins, functinal prperties and tpgraphic representatin. Jurnal f Cmparative Neurlgy 199, VOGELS, R. & ORBAN, G.A. (1991). Quantitative study f striate single unit respnses in mnkeys perfrming an rientatin discriminatin task. Experimental Brain Research 84, ZEKI, S.M. (1974). Functinal rganizatin f a visual area in the psterir bank f the superir tempral sulcus f the rhesus mnkey. Jurnal f Physilgy 236, ZOHARY, E., CELEBRINI, S., BRITTEN, K.H., NEWSOME, W.T. (1994). Neurnal plasticity that underlies imprvement in perceptual perfrmance. Science 263,