Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

Transcription

1 Leigh R. Hochberg et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm Nature, 17 May 2012 Paper overview Ilya Kuzovkin 11 April 2014, Tartu

2

3

4

5

6

7 etc

8 How it works? etc

9 2012 Reach and grasp by people with tetraplegia using a neurally controlled robotic arm Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems 2011 Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia Neuronal ensemble control of prosthetic devices by a human with tetraplegia Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter

10 2012 Reach and grasp by people with tetraplegia using a neurally controlled robotic arm Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems 2011 Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia Neuronal ensemble control of prosthetic devices by a human with tetraplegia Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter

11 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons. Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference

12 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.

13 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.

14 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons. Posterior probability Likelihood Prior probability Hypothesis (hand motion) Evidence (sequence of observed firing rates) Marginal likelihood (can be ignored since it is the same for all hypothesis)

15 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons.

16 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons. Likelihood term models the probability of firing rates given a particular hand motion

17 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons. Likelihood term models the probability of firing rates given a particular hand motion linear Gaussian model could be used to approximate this likelihood and could be readily learned from a small amount of training data

18 uses Bayesian inference techniques to estimate hand motion from the firing rates of multiple neurons. Likelihood term models the probability of firing rates given a particular hand motion linear Gaussian model could be used to approximate this likelihood and could be readily learned from a small amount The prior term defines a of training data probabilistic model of hand kinematics and was also taken to be a linear Gaussian model.

19 Neural Coding

20 Neural Coding of Hand Kinematics

21 Neural Coding of Hand Kinematics

22 Neural Coding of Hand Kinematics

23 Neural Coding of Hand Kinematics Experiment 1: 23/25 neurons are correctly described by equations (4) and (5)! Experiment 2: 39/42 neurons correctly described by (4) and (5)

24 Neural Coding of Hand Kinematics Experiment 1: 23/25 neurons are correctly described by equations (4) and (5)! The relationship between the kinematics of the arm and the behavior of the neurons is strong Experiment 2: 39/42 neurons correctly described by (4) and (5)

25 Learning the model

26 Detour: Multivariate normal distribution

27 Detour: Multivariate normal distribution

28 Detour: Multivariate normal distribution Why covariance matrix and not just a vector of variances?

29 Definitions

30 Definitions

31 Parameters of the model

32 Parameters of the model H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise

33 Parameters of the model H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise A is the relation between the state at time k+1 and the state at time k W is covariance matrix of the noise

34 Parameters of the model H is the relation between the firing rates of each of the neurons and states of the arm Q is covariance matrix of the noise A is the relation between the state at time k+1 and the state at time k W is covariance matrix of the noise Matrices A, H, Q, W is what we want to learn from the training data

35 The Learning

36 Decoding

37 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference Note that now x and z and everything else refer to the test data

38 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference

39 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference The probability that the hand can move in the way it did

40 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference The probability that the hand can move in the way it did The probability that hand can end up in the state where it was in time k-1

41 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference the Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state. (Wikipedia)

42 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference

43 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference

44 Decoding was performed using a Kalman filter which gives an efficient recursive method for Bayesian inference

45 Results

46

47

48 2012 Reach and grasp by people with tetraplegia using a neurally controlled robotic arm Efficient Decoding With Steady-State Kalman Filter in Neural Interface Systems 2011 Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia Neuronal ensemble control of prosthetic devices by a human with tetraplegia Bayesian Population Decoding of Motor Cortical Activity using a Kalman Filter

49

50 The steady-state Kalman filter significantly increases the computational efficiency for even relatively simple neural spiking data sets from a human NIS. < > The decoding complexity is reduced dramatically by the SSKF, resulting in approximately seven-fold reduction in the execution time for decoding a typical neuronal firing rate signal.

51 Summary