Machine Learning for Multi-Cell Human Intent Inference

Transcription

1 Machine Learning for Multi-Cell Human Intent Inference Joshua Jacobs, Jennifer Wortman, Michael Kearns, Michael J. Kahana Departments of Neuroscience, Computer and Information Science, and Psychology University of Pennsylvania Philadelphia, PA 94 Itzhak Fried Division of Neurosurgery University of California at Los Angeles Abstract We applied modern machine learning algorithms to the study of simultaneous firings of neurons in human subjects implanted with intracranial electrodes as part of a surgical treatment for drug-resistant epilepsy. These subjects played a taxi-driver video game in which they alternated between searching for passengers, and delivering these passengers to their destination. We used support vector machines and boosting to create classification models that use multi-cell neural activity to predict whether subjects were searching for a passenger or a store. Despite methodological challenges (such as the electrodes being positioned for clinical purposes, not for our classification task), these models were able to frequently and significantly outperform a number of natural baseline (single-cell) classifiers. The SVM models are stronger than those of boosting on our problem. Furthermore, model accuracy is significantly enhanced when we consider the confidence levels output by the models (i.e. when we choose to classify only a high-confidence subset of the test data). We present compelling evidence that the learned models are integrating information from multiple cells. This work demonstrates the potential of machine learning algorithms for isolating correlates of behavior in complex neural datasets, and is one of the few such applications to human neural data for a relatively abstract cognitive state such as intent. Introduction The activity of individual neurons in the mammalian brain has been shown to carry information about an animal s past experiences, current perceptions, and future actions. Although much has been learned about the behavioral correlates of neural activity in a range of species, the functional role of neurons underlying human cognition has only begun to be characterized. In the present work, we recorded neural activity in human subjects who had electrodes

2 implanted for the treatment of their drug-resistant epilepsy. The clinical team determined the placement of the electrodes in order to identify the sites of seizure onset for subsequent resection surgery. While they were fitted with these electrodes, we asked subjects to play Yellow Cab, a virtual taxi-driver game that involved alternating between the two highlevel goal states of searching for passengers and subsequently delivering the passengers to specified destinations. We used two standard classification algorithms, boosting and support vector machines (SVMs), to create models of each subject s high-level goal state from their neural activity. These models attempt to predict whether subjects are looking for a passenger or a store based on their neural firing patterns. Because we observe different neurons in each subject, these models are fit separately for each Yellow Cab session. We made no assumptions about which cells carry information about subject s goal state, nor about correlations between cells; we relied on the classification algorithms to discover this information. In some subjects, these algorithms were able to predict human behavior with high accuracy. Models in which groups of neurons were analyzed simultaneously displayed higher predictive power than models utilizing information about only a single neuron. Furthermore, models were able to make better predictions when only required to predict on highconfidence subsets of test data. These findings indicate that machine learning is a powerful tool for studying multi-cell patterns of neural firing and neural correlates of cognition. 2 Background Previous work has suggested that activities of single neurons correlate with behavior during spatial navigation. Cells that are informative of position (termed place cells ) have been demonstrated in rodents [], monkeys [2], and humans[3]. Furthermore, it has been shown that activity of individual neurons is correlated with a subject s destination during navigation [3]. Although most work characterizing neural responses during navigation studied cells individually, theories of population coding suggest behavior is best predicted by simultaneously analyzing multiple cells [4]. Previous work using machine learning with neural data included work predicting human cognitive state with functional magnetic resonance imaging [5]. The issue of position reconstruction in rodents during a navigation task was addressed in [6] and [7]. More recent work has addressed similar problems using more sophisticated learning algorithms. For example, firing rates of individual neurons in the monkey visual cortex were used to predict the orientation of a visual stimulus using a support vector machine algorithm [8]. The work we describe is perhaps the first neuroscience application of machine learning to the inference of a cognitive state as abstract as the subject s intended goal. 3 Methodology 3. Behavioral Methods In Yellow Cab, subjects navigated a virtual town either using keyboard arrow keys or a hand-held joystick. The town consisted of 3 clearly labeled target stores and 6 unlabeled buildings (see Figure ) arranged in a 3 3 grid. Passengers were picked up by driving near them. After pickup, a text screen informed the subject of the destination store to which the passenger should be delivered. Subjects were motivated to complete each task quickly because they received points upon successful passenger delivery, and had points deducted for every second of game-play. During the course of a 2-4 minute experimental session, passengers completed either 2 or 45 pickup/delivery cycles. See [9] for more details on this task.

3 Figure : A. Map of the virtual town. B-D. Images of example subject destination stores, as seen during navigation. 3.2 Human Electrophysiological Recordings Each electrode was terminated with a set of 4-µm platinum-iridium microwires. Signals from the microwires were sampled at 4-32kHz and action potentials of individual cells were isolated based on spike waveform and distribution of inter-spike intervals []. Electrodes overlying brain regions exhibiting seizure-related activity were excluded from analyses. We recorded from 3 subjects who participated in a total of 28 sessions; individual subjects performed between -3 sessions. In each session we recorded between 3-38 individual neurons. All patients provided informed consent for testing, and all studies conformed with the guidelines of the Medical Institutional Review Board at UCLA. Electrode implantation is dictated by clinical purposes, so different brain areas were observed in each patient. Frequently recorded brain regions include the hippocampus, parahippocampal region, amygdala and frontal lobes. Furthermore, because electrode position can shift over time and multiple sessions of a single subject often occur on different days, it is possible that two recordings from the same electrode sample different neurons. These factors led us to treat each testing session individually in subsequent analyses. It is also worth bearing in mind that since the electrodes are not placed for the purposes of our classification task, it is to be expected (and it occurs) that for some sessions and subjects, the neural firings simply will not contain information relevant to our classification task. 3.3 Binning into Temporal Epochs To prepare the multi-cell neural data for the classification task, we divided each session into fixed-length temporal epochs. For each epoch, the firing rate of each cell was calculated, and a label was assigned according to whether the subject was looking for a passenger or store during the corresponding time period. If the goal-searching state changed in the middle of a temporal epoch, the current epoch was combined with the previous epoch, and a new epoch began at the point of transition. The number of passenger- and storesearching epochs varied across sessions because the length of time spent in epoch type varied according to subject navigation proficiency and style. A typical session lasted 3 minutes, so this would lead to approximately 8 one second epochs, 8 ten second epochs, and so on. 3.4 General Classification Methodology Classification tasks were performed seperately for each session of data. Since the data were limited, especially at longer epoch lengths, a cross-validation methodology was used to estimate model accuracy. Each learning algorithm was run 3 times on every data set (session). For each run, 5% of the epochs were randomly set aside for testing; the remaining 85% were used for training the predictor. Test error rates were estimated and reported as the mean test error over the 3 trials.

4 Average error rate Naive.5 SVM Single Cell Epoch size (seconds) Figure 2: Epoch size versus average error rate, averaged over all subjects. To assess the performance of our prediction, we compared the performance of SVMs and boosting with two baseline classifiers. We created a naive classifier which always predicts the output label value that occurred most frequently in the training dataset. Because of our interest in the potential value of multi-cell models, we also compared to a stronger baseline that searches the training dataset to find the single-cell firing rate and threshold that best predicts the label. In all cases we compared the test errors of our learned models with those of the naive and best single-cell classifiers. Support vector machine models were created using version 6. of SVMlight [] with default options and a linear kernel; early experimentation with a quadratic kernel showed no improvement over the linear model. models were generated using BoosTexter [2] version 2., again with default parameter settings. BoosTexter uses boosting on top of simple decision stumps, each of which is a threshold function of a single cell s firing rates. 4 Results 4. Initial Findings and Selection of Optimal Epoch Length To analyze the power of classification techniques for predicting human intent, it is necessary to determine the most useful way to discretize continuous spike train data into feature vectors. An obvious choice is to let each data instance be a vector containing the firing rates of each cell over a given period of time. The choice of an optimal length of time for each of these periods must take into consideration the data quantity and quality. Previous experiments [3, 4] with primates were able to perform accurate predictions using firing rates averaged over periods of ms to second, but these studies were able to position recording electrodes in brain regions known to respond in a task-specific manner. By comparison, in the present dataset, electrodes are positioned clinically, so it is likely that data is noisier with respect to any behavioral variable. To address this issue, we varied the scale of temporal epoching, to determine the best way of extracting task-related information. There is a trade-off to consider here, as larger time epochs will result in a smaller number of available training instances when the data is sparse and might miss subtleties of patterns in spike rates. Thus, we begin by exploring the capabilities of learning algorithms over a wide range of epoch sizes. Figure 2 shows the average error rate over all sessions with the data discretized into epochs

5 Single cell SVMs Single cell SVMs Error rate Error rate Fraction Predicted Fraction Predicted Figure 3: Error rate as a function of fraction of data predicted, for two sample subjects. At each fraction, the predictor chooses to predict the epochs on which it is most confident. of various lengths. It is clear from this graph that boosting and SVMs both have some level of predictive power in this task, as both models outperform the naive baseline at all epoch lengths (p <.5, paired t-test). SVMs also achieve lower error rates than boosting. However, the above alone is not enough to claim that either classification algorithm is fully exploiting the variety of cells available. Although SVMs do outperform the best single-cell classifier at lower time epochs, this difference in performance is not statistically significant. In a moment we provide statistically strong evidence that SVMs are indeed learning a multi-cell model whose predictive accuracy outperforms any single-cell model, when we incorporate model confidence. Since support vector machines and boosting both exhibit their lowest absolute error rates at an epoch length of nine seconds (see Figure 2), epoch lengths will be fixed at nine seconds for the remainder of the experiments discussed. 4.2 Prediction Accuracy and Model Confidence It is possible that classification models are able to predict by recognizing the presence of specific patterns of neural activity that occur only sometimes during certain goal states. This would result in high prediction accuracy when this pattern is recognized, and lower accuracy at other times. In such a case, we might hope that the model s prediction confidence would be highest for epochs whose firing closely matches a recognized pattern. We examined this hypothesis by looking at the relation between prediction accuracy and model confidence, in the spirit of ROC curves or precision-recall analyses. In addition to predicting a label for each epoch, the models output by SVMs and boosting each produce a measure of their confidence in that prediction. The model generated by the support vector machine algorithm is of the form sign(w x+b) where x is the input vector, w is a linear combination of support vectors, and b is an added bias term. A positive sign predicts one label (e.g. passenger-searching), while a negative sign would cause the epoch to be classified with the other label (e.g. store-searching). It is thus possible to interpret w x + b as a measure of how confident the model is that its label prediction for a given instance is correct. Similar confidence measures can be computed for boosting models. Confidence of the single-cell baseline model was computed as the distance between the firing rate of the single cell chosen and the model s threshold value. Figure 3 shows the prediction error rates for two example subjects when our support vector machine model is used to predict labels for data subsets. With this analysis, we are effectively using the SVM model to answer the question: For a given test dataset, choose the N% of the epochs that you can predict most confidently, and report your error rate. In these

6 .5.45 Error rate Single cell SVM.75.5 fraction predicted.25 p. of baseline performance.5... SVM.75.5 fraction predicted.25 Figure 4: Error rate as a function of fraction of data predicted, across all subjects. Left panel shows average performance of each predictor when asked to predict fixed fractions of data. Right panel shows the probability of identical performance of the learned models (SVMs and boosting) and the best single-cell model (paired t-test). SVMs significantly (p <.5) outperform the single-cell predictor when the models are asked to predict on 8% or less of the test data. SVM error rate % Single cell error rate SVM error rate 5% Single cell error rate SVM error rate 25% Single cell error rate Figure 5: Error rates from using SVMs to predict the subset of each test dataset that it is most confident on. Each point represents the single-cell and SVM error rate for a single session. Sessions that are below the diagonal had higher prediction accuracy with SVMs than with the single-cell predictor. Diagonal line denotes equal performance of SVM and single-cell predictors. subjects, SVM prediction accuracy increases dramatically in epochs where the predictor is confident. In Figure 4, performance on this task is summarized across all subjects. Here it is seen that classification performance increases when only predicting epochs when it has a high prediction confidence. Figure 4(left) shows that the SVM predictor achieves the best performance increase when given the freedom to choose only a subset of the data to classify. As shown in figure 4(right), the SVM predictor achieves performance that is better than the single-cell predictor at a statistically significant level when it chooses 8% or less of the test data to predict. To further detail the performance of the SVM predictor, in Figure 5 we show the SVM and single-cell error rates for each session when used to predict for various fractions of the test dataset. 4.3 Further Analysis of the Models SVMs and boosting are capable of creating models that use either small or large numbers of cells. Figure 6(left) illustrates the distribution of cell influences (magnitudes of coefficients)

7 Fraction of cells SVM Cell Influence Fraction observed Influential cells Total cells.5 Fraction of max. influence Cell Influence H PR A Fr? Region Figure 6: Left: Fraction of cells exceeding an influence threshold in SVM model. Threshold (abscissa) is computed as a fraction of the largest influence of any cell in the model. Center: Correlations between the models chosen by each algorithm on a sample data set. Each point represents the influence of one single cells. Right: Brain-region grouped fraction of most influential cells in SVM model, and total observed cells. Region key: H - Hippocampus, PR - Parahippocampal region, A - Amygdala, Fr - Frontal lobes,? - other regions. in the SVM models. This shows that roughly 25% of cells in each model have at least half of the influence of the maximum cell, indicating that the SVM models are indeed relying on information from multiple cells, since most of our sessions recorded from dozens of cells. Any neural correlates of goal state that exist in the data should be visible in predictive models created by both boosting and support vector machines. It is informative to compare the influence of each cell in the models generated by these algorithms. For boosting models, we define the influence of a cell as the difference between the weight that boosting places on a cell when the cell is maximally firing, and the weight that is placed on the cell when it is silent. To facilitate comparison of influences between SVMs and boosting, we normalize influences within a model by the sum of all influences. Figure 6(center) illustrates the correlations between normalized measures of cell influence in boosting and SVM models for one dataset. This plot shows that while individual cells differ in how they affect prediction, there is a large degree of consistency in the weights assigned to each cell across boosting and SVMs. Across all sessions of data, the average correlation between the two is.7295, and with the exception of one data set containing information on only three cells, no data sets show negative correlations. As would be expected, the error rates achieved on each session of data by boosting and support vector machines are also related, with a correlation of.884. Thus not only are the same cells influencing each model in positive or negative ways, but the models are showing the good and bad performance on the same subjects, further indicating that some data sessions inherently contain more useful data for goal prediction than others. Finally, we were interested in determining which brain regions are most responsible for our models predictive power. In Figure 6(right) we examine this in terms of which brain region contains the cell with highest influence in the SVM model for each session. Here we observe that cells in the amygdala have the highest influence in 46% of all SVM models, while comprising only 26% of overall cells. The high frequency of maximal model influence indicates that amygdalar neurons are more indicative of goal-searching behavior than other brain regions (p <., binomial test).

8 5 Conclusions We have shown that machine learning can be a useful tool for studying the correlation between cell firing rates and high-level human behavior by generating predictive models while making few assumptions about the nature of the underlying data. We have shown that this predictive power stems from the ability to study groups of multiple cells, and that in some subjects, when prediction confidence is high, prediction accuracy is nearperfect. This indicates the presence of distinct patterns of neural activity that correlate with behavioral states. The observation that the neurons in the amygdala are especially predictive of subject s goal-searching status is consistent with observations of amygdalar involvement in emotional processing. References [] J. OKeefe and J. Dostrovsky. The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat. Brain Research, 34:7 75, 97. [2] T. Ono, K. Nakamura, H. Nishijo, and S. Eifuku. Monkey hippocampal neurons related to spatial and nonspatial functions. Journal of Neurophysiology, 7:56 529, 993. [3] A. Ekstrom, M.J. Kahana, J.B. Caplan, T.A. Fields, E.A. Isham, E.L. Newman, and I. Fried. Cellular networks underlying human spatial navigation. Nature, 425:84 87, 23. [4] A.P. Georgopoulos and Kettner R.E. Schwartz, A.B. Neuronal population coding of movement direction. Science, 233:46 49, 986. [5] T. M. Mitchell, R. Hutchinson, R. Niculescu R. S., Pereira, X. Wang, M. Just, and S. Newman. Learning to decode cognitive states from brain images. Machine Learning, 57:45 75, 24. [6] O. Jensen and J. E. Lisman. Position reconstruction from an ensemble of hippocampal place cells: contribution of theta phase coding. Journal of Neurophysiology, 83: , 2. [7] K Zhang, I Ginzburg, BL McNaughton, and TJ Sejnowski. Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. Journal of Neurophysiology, 79:7 44, 998. [8] J. Eichhorn, A.S. Tolias, A. Zien, M. Kuss, C. E. Rasmussen, J. Weston, N.K. Logothetis, and B. Scholkopf. Prediction on spike data using kernel algorithms. Advances in Neural Information Processing Systems, 6: , 24. [9] J. B. Caplan, J. R. Madsen, A. Schulze-Bonhage, R. Aschenbrenner-Scheibe, E. L. Newman, and M. J. Kahana. Human theta oscillations related to sensorimotor integration and spatial learning. Journal of Neuroscience, 23: , 23. [] K.D. Harris, D.A. Hense, J. Csicsvar, H. Hirase, and G. Buzsaki. Accuracy of tetrode spike separation as determined by simultaneous intracellular and extracellular measurements. Journal of Neurophysiology, 84:4 44, 2. [] T. Joachims. Making large-scale SVM Learning Practical. Advances in Kernel Methods - Support Vector Learning. B. Scholkopf and C. Burges and A. Smola (ed.), MIT-Press, 999. [2] Robert E. Schapire and Yoram Singer. Boostexter: A boosting-based system for text categorization. Machine Learning, 39(2/3):35 68, 2. [3] L. Shpigelman, Y. Singer, R. Paz, and E Vaadia. Spikernels: Precicting arm movements by embeding population spike rate patterns in inner-product space. Neural Computation, 25. [4] I. Gat, N. Tishby, and M. Abeles. Hidden markov modeling of simultaneously recorded cells in the associative cortex of behaving monkeys. Network: Comput. Neural Syst, 8: , 997.