A Comparison of Monkey Afferent Nerve Spike Rates and Spike Latencies for Classifying Torque, Normal Force and Direction

Transcription

1 A Comparison of Monkey Afferent Nerve Spike Rates and Spike Latencies for Classifying Torque, Normal Force and Direction Stephen J. Redmond *, Antony W. Goodwin, Nigel H. Lovell *, Ingvars Birznieks * University of New South Wales, Sydney, NSW 252, Australia Prince of Wales Medical Research Institute, Sydney, NSW 23, Australia Department of Anatomy and Cell Biology, University of Melbourne, Australia s.redmond@unsw.edu.au, a.goodwin@unimelb.edu.au, n.lovell@unsw.edu.au, i.birznieks@powmri.edu.au Abstract The movement of primate hands and limbs is governed by a biological control system which utilizes information from tactile sensory afferents. However, it is not entirely understood how complex stimuli, such as force, direction and torque applied to the finger pad, are encoded within the population responses of these afferent mechanoreceptors. This study investigates whether first spike latency after the onset of a torque stimulus, superimposed on an applied normal force, from a population of 25 fast-adapting type-i (FA-I) afferents, can achieve a comparable classification accuracy (using a Parzen window classifier) as the more traditional feature of spike count. Results show that FA-I afferents perform poorly at identifying the three levels of static normal force applied, but achieve % recognition rate within 5 ms when classifying two different magnitudes and directions of torque, with an accuracy of over 9% within 3 ms of the torque onset. This is in line with other reported studies which indicate that afferent first spike latency contains similar information about tactile stimuli as spike count. I. INTRODUCTION The movement of primate hands and limbs is governed by a biological control system which utilizes information from tactile sensory afferents. However, it is not entirely understood how complex stimuli, such as force, direction and torque applied to the finger pad, are encoded within the population responses of these afferent mechanoreceptors. Performing seemingly mundane manipulation of delicate objects, such as grasping, lifting and rotating, becomes a nontrivial task in the absence of tactile sensory input []. Tactile afferents encode critical manipulative parameters, which occur during simple object manipulation, like the grip force or normal force applied to the object to prevent slip and/or rotation; achieving the latter implies that a torque is also being applied. Torque is ubiquitous in everyday tasks and arises when an object s centre of mass does not align with the vector of force application [2, 3]. Human experiments employing a two finger opposition grip [4-6] and tasks where objects are held in a multi-finger grasp [7-] have been used to investigate sensorimotor mechanisms controlling the effects of torque during object manipulation. Predictive strategies based on internal representations which must be updated by sensory inputs, have been shown to be largely responsible for smooth self-paced object manipulation and coordination of fingertip forces [2]. An adequate reaction can only be triggered when errors or unpredictable events occur in the sensory input, which do not align with predicted events. We have recently investigated how such information is encoded in the activity generated by the afferent neurons, or mechanoreceptors, in the finger pads of monkeys [3, 4]. In these studies, a Parzen window classification model was applied to the data ensemble, consisting of accumulated spike counts (summing from the onset of the stimulus) to investigate how accurately the various stimulus attributes, of normal force, torque and direction, were encoded in the population of afferent nerve responses. These analyses attempted to solve how interactions between numerous stimulus features may be disentangled in real-time. While studies, like those mentioned above, have shown that features of the afferent nerve response, such as accumulated spike count, contain relevant information to perform the realtime classification of multiple simultaneous stimuli parameters, others have demonstrated that features, such as the first spike latency also contain the same (if not more) information about the same stimuli [5, 6]. The aim of this study is to compare the real-time classification performance when using spike count features against first spike latency features to discriminate between the normal force and torque applied to the finger pad, and the direction which the torque is applied. The same methodology is applied as was used in [3, 4], which is summarized in the following sections. II. METHODS A. Neural recordings Recordings of afferent nerve responses were made from three anesthetized Macaca nemestrina monkeys (weight: 4.5 to 7.6 kg), with approval from the University of Melbourne Ethics Committee and conforming to the National Health and Medical Research Council of Australia s Code of Practice for non-human primate research APSIPA. All rights reserved. 72 Proceedings of the Second APSIPA Annual Summit and Conference, pages , Biopolis, Singapore, 4-7 December 2.

2 After inducing surgical anesthesia, single cutaneous mechanoreceptive afferents in the median nerve were isolated and recorded from, using standard procedures [7]. Using established criteria, the responses of each afferent to static and rapidly changing stimuli, and the receptive field size, was analyzed [8]. Calibrated von Frey hairs were used to determine the location of each receptive field. Using the results of this analysis of afferent response characteristics and receptive field area, each single fiber was labeled into wellknown categories of slowly adapting type-i (SA-I), fastadapting type-i (FA-I), or fast-adapting type-ii (FA-II) afferents. All 25 responding FA-I afferents innervating the glabrous skin of the distal phalanx of digits 2, 3 and 4 were included in this study; the response of FA-II afferents was not reliable. It is not instructive to use SA-I afferents, since we wish to examine first spike latency after the onset of the torque stimulus, however, the background normal force applied (discussed below) will cause the SA-I afferents to continuously fire, making them unsuitable for first spike latency analysis. In order to securely affix the digits, they were splayed and the dorsal aspect of the hand was embedded in plasticine up to the mid-level of the middle phalanges. The distal phalanges were stabilized by gluing the fingernails to small metal plates which were fixed to a post embedded in the plasticine. The fingertip was free to deform as the glabrous skin of the distal phalanges was not in contact with the plasticine (see Methods in [9]). B. Stimulation procedure ) Stimulator: The same stimulation procedure was applied as in [3, 4], using a custom-made mechanical stimulator, controlled using LabVIEW 5 software (National Instruments, Austin, TX), applying concurrent normal force and torque forces. The forces and torques were measured using a six-axis force-torque transducer (Nano FT; ATI Industrial Automation, Apex, NC). The resolution of the transducer to force was.25 N. The torque resolution was.625 mnm. The stimulus applicator was a flat circular surface (diameter 24 mm) covered with fine grain sandpaper (5 grade). 2) Application of torsional loads: The stimulus was applied to the center of the flat portion of the volar surface of the fingertip on digits 2, 3 and 4. The normal force direction of application was at right angles to the skin surface. Three normal force magnitudes (.8, 2.2 and 2.5 N) and two torque magnitudes (2. and 3.5 mnm) were applied in both clockwise and anticlockwise directions. This gave 2 (3 2 2) force, torque, direction combinations. For each normal force, each torque was applied in ascending order. Once both torque values were applied for a given normal force, the normal force was increased and the torques were reapplied. This was repeated until all three normal forces had been applied for both torque values. This entire process was repeated six times, after which the entire procedure was repeated six more times with the torque applied in the opposite direction. In summary, there were 2 different stimulation combinations, each applied 6 times in total, giving 72 different training examples for the classifier model (described later), when the responses of all afferents were considered as a simultaneously acquired ensemble. For details on the time evolution of the normal force and torque, the reader is referred to [3]. A summary of the force/torque application process is as follows. The normal force was ramped up during a s loading phase; the normal force was held constant for 3.6 s of the plateau phase, followed by a s retraction phase removing the normal force. The application of the torque commenced s after the plateau phase of the normal force had begun and was superimposed on this constant normal force. The torque was loaded over s, held constant at its maximum value for.5 s, and unloaded over a final s duration. Therefore, the normal force was completely retracted s after the unloading of the torque force. C. Classification features A classification of the stimulus parameters (normal force, torque and direction) was performed at ms intervals after the onset of the torque. The classifier model design is described in later sections. This section describes the features used for classification. There are two types of features under comparison; accumulated spike count and first spike latency, as outlined in the following sections. ) Accumulated spike count: The accumulated spike count is simply the total number of spikes which have occurred since the onset of the torque application, evaluated in ms intervals. 2) First spike latency: There are different ways which the first spike latency, τ, could be represented, since before and after the occurrence of the first spike some feature value must be assigned to each afferent response, as dealing with missing features is problematic when training classifier models [2]. Therefore two different feature values are investigated for each afferent to represent the first spike latency: (i) the feature takes the value of before the first spike occurs and τ after; (ii) the feature value is before the first spike occurs and afterwards (acting as a binary indicator of whether the nerve has fired yet, or not). D. Stimulus classification using afferent populations ) Feature vector ensemble: As described in Section II.B.2, there were 2 stimulus combinations applied. Each combination of force, torque and direction was applied six times, leading to 72 training patterns, or feature vectors. Each stimulus combination, i, leads to a 25 element feature vector x i. Each element the feature vector is a spike count, or first spike latency value, from one of the 25 FA-I afferents, as described in Section II.B.2. Note that while only one afferent is recorded from at any one time, the feature vector is constructed as an ensemble, as if all afferents were recorded simultaneously. 2) Parzen window classifier model: A Parzen window classifier model was used to discriminate between each of the stimulus parameter classes [2]. There are three classes for normal force and two for each of torque magnitude and torque 72

3 before, τ after - before, after (a) Normal force Torque magnitude Torque direction (b) - before, τ after - before, after (c) - before, τ after - before, after Fig.. Classification performance using two different first spike latency features. (a) Classification accuracy for normal force, (b) torque and (c) direction of torque application. Classification is performed at ms intervals after the start of torque application. The two different latency features are described in Section II.C.2. Chance performance would be 3, and, respectively, for the three plots. direction, corresponding to each of the stimulus values applied (see Section II.B.2). For example, in the case of classifying normal force there are three classes ω {.8, 2.2, 2.5} N. Separate independent classifiers are designed for the different classification tasks of normal force, torque and direction. Equal a priori probabilities are assumed for each class of each stimulus classification task. A multivariate normal Gaussian window is used for the Parzen window classifier. The radius variable for the Gaussian window, r, which serves to adjust the smoothness of the nonparametric density estimation, is given the value r= after each afferent response feature has been normalized across all training vectors, by removing the mean and dividing by the standard deviation. 3) Cross-fold validation: A leave-one-out cross-fold validation scheme was employed to obtain an unbiased estimate of generalized classification performance [2]. After removing a single test vector from the training data the remaining 7 feature vectors were used to train the Parzen window classifier. The withheld vector was then reintroduced for testing once training was complete. This process was repeated for all 72 training patterns. The results presented below represent the percentage of these 72 test patterns which were correctly classified. E. Classification comparisons ) Comparison of latency features: In order to determine which representation of first spike latency achieves the best classification performance, the two proposed features for representing spike latency are compared. 2) Comparison of spike count and first spike latency: Using the best performing spike latency feature, a comparison is performed between the discriminative powers of first spike latency and accumulated spike count. 3) Combined performance of spike latency and count: Finally, the combined classification accuracy achieved when simultaneously using spike count and latency for the classification task is investigated by appending the two feature vectors together, to give a 5 element feature vector. III. RESULTS A. comparison results Fig. (a), (b) and (c) show the plots of classification accuracy for normal force, torque magnitude and torque direction, respectively. On each graph, the classification accuracy, assessed at ms intervals is shown when using each of the two proposed latency features, described in Section II.C.2. The graphs assess the epoch of time from the onset of the torque until 2 ms after the torque has reached its maximum and is being held constant. We note from Fig. that the first spike latency representation which uses a zero feature value before the occurrence of the first spike, and the latency time afterwards, provides the maximum classification accuracy for two of the three classification tasks at 7 ms (2 ms after the torque has reached its plateau), with the classification performance for torque only marginally worse than using the binary indicator feature. B. versus spike count results Fig. 2 illustrates the comparative performance of the accumulated spike count, and the best first spike latency feature, which uses the latency time as a feature value after the first spike is generated for that afferent. C. Combined latency and count performance 722

4 Fig. 2 also contains a plot of the classification accuracy obtained using a feature vector which employs both spike count and first spike latency. IV. DISCUSSION Examining Fig. we see that there is little discriminative capacity in the first spike latency before 25 ms. After 25 ms, the classification accuracy begins to improve for all classification tasks. For normal force, the classification accuracy does not improve much beyond chance, even when 7 ms is reached. At this point the spike latency measure, (a) Normal force Torque magnitude Torque direction (b) (c) Fig. 2. Comparison of the classification performance of the first spike latency (time elapsed before first spike) and the accumulated spike count on the three classification tasks of (a) normal force, (b) torque magnitude and (c) torque direction. features allow close to % classification accuracy of torque parameters 7 ms after the stimulus onset. Chance performance would be 3, and, respectively, for the three plots. which fixes the feature with the spike latency after the first spike occurs, provides the best performance of both proposed latency features, for all three classification tasks normal force, torque and direction. The response for FA type afferents is not expected to be crucial for discriminating normal force in our analyzed dataset, as FA afferents only respond to dynamically changing features of stimuli, and at the onset of torque application the normal force has already been at its plateau for s, and thus does not per se excite FA afferents. However, normal force information theoretically could still be extracted based on the interaction effect normal force might have on afferent responses to torque; for instance, if normal force was to somehow modulate the FA afferents sensitivity to torque. latency features perform similarly when classifying torque magnitude and direction, obtaining high accuracies by approximately 35 ms after the stimulus onset. Interestingly, using a simple binary indicator feature, which indicates if the afferent has fired at least once since the onset of the torque, shows very similar performance in the early stages of the recognition task (around 25 ms), when compared to the other latency feature, which contains more detailed timing information. Gautrais and Thorpe were the first to identify information encoded in temporal activation order for the visual system [2] although the usefulness of afferent recruitment order was first shown by Johansson and Birznieks [5]. Also, examining the time around 7 ms, when the binary latency feature shows good performance, illustrates that the final activation pattern of the afferents contains enough information to complete the classification task. The dip in classification performance for this feature, around 38 ms when classifying torque magnitude, may be an indicator that the order in which the afferents are activated during this period varies from trial to trial. Since the binary feature removes the actual latency timing information (and only codes the population activation distribution) it is too difficult to perform the discrimination task using so little information during this dynamic phase. Fig. 2 reproduces the curves shown in Fig., for the latency feature, in addition to the performance achieved when using a more traditional feature of the afferent responses the accumulated spike count. The similarity of the curves shows that first spike latency contains enough information to perform the classification task with a comparable accuracy to using spike count. Examining both Fig. (b) and Fig. 2 (b) together, one anomaly to note is the dip in accuracy which occurs around 4 ms. This happens when using the spike count feature (Fig. 2 (b)) and a similar dip in performance is observed in Fig. (b), when using the binary indicator of whether the afferent had fired, or not, at each time interval. This might imply that in fact spike latency is a more robust feature for classifying the stimulus parameters used, when compared to population activation features, of which the binary latency feature and the spike count are approximate surrogates. Finally, also shown in Fig. 2 are results illustrating that combining both the spike latency and spike count features and 723

5 repeating the classification tasks demonstrates no discernable performance improvement over using either feature alone. This indicates that the latency and count contain mostly the same information about the stimulus. However, a bootstrap analysis is required to verify this claim. That is not to say that this will always be the case for all stimulus events not tested here; however, within the constraints of the stimuli applied here, both contain adequate information to perform the classification tasks of recognizing torque magnitude and direction. Similar studies of first spike latency have been performed and appear to concur with the results presented here. An excellent paper by Saal et al. analyzes and compares the entropy information contained in spike latency and spike counts, for single afferent responses under similar stimulation conditions (although torque is not used). They show the information content in spike latency is 2.2 and.6 times that of spike count for the discrimination tasks of determining applicator curvature and force direction, respectively [6]. However, selecting features for classification tasks based on entropy information measures is known to provide no guarantee of improved classification performance, due to what may possibly be complex correlations between features. As a result, there is a sizeable literature on the topic of feature selection for pattern classification which attempts to address this issue [22, 23]. V. CONCLUSION This study has demonstrated that first spike latency after the onset of a torque stimulus, superimposed on an applied normal force, from a population of 25 FA-I afferents recorded from Macaca nemestrina monkeys, achieves similar classification accuracy as using the more traditional feature of spike count to quantify the afferent population response. FA-I afferents perform poorly at identifying the three levels of static normal force applied, but achieve almost % recognition rate on two different magnitudes and directions of applied torque within 5 ms, and an accuracy of over 9% within 3 ms. This is in line with other reported studies which indicate that afferent first spike latency contains similar information about tactile stimuli as the spike count. VI. ACKNOWLEDGEMENTS This research was supported by the National Health and Medical Research Council (NHMRC) and Australian Research Council (ARC) Thinking Systems grant TS The neurophysiology experiments were performed at the Department of Anatomy and Cell Biology, University of Melbourne, Australia. We would like to thank Dr Heather Wheat and Dr Lauren Salo for their contribution to the collection of the neurophysiological data. REFERENCES [] J. R. Flanagan, M. C. Bowman, and R. S. Johansson, "Control strategies in object manipulation tasks," Curr. Opin. Neurobiol., vol. 6, pp , 26. [2] R. S. Johansson and G. Westling, "Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects," Exp. Brain Res., vol. 56, pp , 984. [3] H. Kinoshita, L. Bäckström, J. R. Flanagan, and R. S. Johansson, "Tangential torque effects on the control of grip forces when holding objects with a precision grip," J. Neurophysiol., vol. 78, pp , 997. [4] A. W. Goodwin, P. Jenmalm, and R. S. Johansson, "Control of grip force when tilting objects: effect of curvature of grasped surfaces and of applied tangential torque," J. Neurosci., vol. 8, pp , 998. [5] A. M. Wing and S. J. Lederman, "Anticipating load torques produced by voluntary movements," J. Exp. Psychol. Hum. Percept. Perform., vol. 24, pp , 998. [6] R. S. Johansson, J. L. Backlin, and M. K. O. Burstedt, "Control of grasp stability during pronation and supination movements," Exp. Brain Res., vol. 28, pp. 2-3, 999. [7] M. K. O. Burstedt, R. Flanagan, and R. S. Johansson, "Control of grasp stability in humans under different frictional conditions during multidigit manipulation," J. Neurophysiol., vol. 82, pp , 999. [8] J. R. Flanagan, M. K. O. Burstedt, and R. S. Johansson, "Control of fingertip forces in multi-digit manipulation," J. Neurophysiol., vol. 8, pp , 999. [9] M. L. Latash, J. K. Shim, F. Gao, and V. M. Zatsiorsky, "Rotational equilibrium during multi-digit pressing and prehension," Motor Control, vol. 8, pp , 24. [] J. K. Shim, M. L. Latash, and V. M. Zatsiorsky, "Prehension synergies in three dimensions," J. Neurophysiol., vol. 93, pp , 25. [] J. K. Shim, J. Park, V. M. Zatsiorsky, and M. L. Latash, "Adjustments of prehension synergies in response to self-triggered and experimentertriggered load and torque perturbations," Exp. Brain Res., vol. 75, pp , 26. [2] D. A. Nowak, S. Glasauer, and J. Hermsdörfer, "Grip force efficiency in long-term deprivation of somatosensory feedback," Neuroreport, vol. 4, pp , 23. [3] I. Birznieks, H. E. Wheat, S. J. Redmond, L. M. Salo, N. H. Lovell, and A. W. Goodwin, "Encoding of tangential torque in responses of tactile afferent fibres innervating the fingerpad of the monkey," The Journal of Physiology, vol. 588, pp , 2. [4] S. J. Redmond, I. Birznieks, N. H. Lovell, and A. W. Goodwin, "Classifying Torque, Normal Force and Direction using Monkey Afferent Nerve Spike Rates," in Eurohaptics Amsterdam, Holland, 2 (in press). [5] R. S. Johansson and I. Birznieks, "First spikes in ensembles of human tactile afferents code complex spatial fingertip events," Nat Neurosci, vol. 7, pp. 7-77, 24. [6] H. P. Saal, S. Vijayakumar, and R. S. Johansson, "Information about Complex Fingertip Parameters in Individual Human Tactile Afferent Neurons," The Journal of Neuroscience, vol. 29, pp , 29. [7] A. W. Goodwin, A. S. Browning, and H. E. Wheat, "Representation of curved surfaces in responses of mechanoreceptive afferent fibers innervating the monkey's fingerpad," J. Neurosci., vol. 5, pp , 995. [8] A. B. Vallbo and R. S. Johansson, "Properties of cutaneous mechanoreceptors in the human hand related to touch sensation," Hum. Neurobiol., vol. 3, pp. 3-4, 984. [9] I. Birznieks, P. Jenmalm, A. W. Goodwin, and R. S. Johansson, "Encoding of Direction of Fingertip Forces by Human Tactile Afferents," J. Neurosci., vol. 2, pp , 2. [2] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification (2nd edition), 2 ed.: John Wiley & Sons, 2. [2] R. V. Rullen and S. J. Thorpe, "Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex," Neural Comput., vol. 3, pp , 2. [22] N. Abe and M. Kudo, "Entropy Criterion for Classifier-Independent Feature Selection," in Knowledge-Based Intelligent Information and Engineering Systems, 25, pp [23] P. Pudil, J. Novovičová, and J. Kittler, "Floating search methods in feature selection," Pattern Recogn. Lett., vol. 5, pp. 9-25,