The Evolution of Neuroprosthetics

Transcription

1 The Evolution of Neuroprosthetics National Academy of Engineering, Google, 2011 Eric C. Leuthardt MD Assistant Professor of Neurological Surgery & Biomedical Engineering Director, Center for Innovation in Neuroscience and Technology website: Blog:

2 Neuroprosthetics: A Brain Computer Interface, or BCI, is device that can monitor and decode the electrical signals of the user's thoughts and convert that information into some type of overt machine control Leuthardt et al, Neurosurgery, 2006

3 Signal Detection

4 Signal Detection Cortex

5 Signal Detection Dura Cortex

6 Signal Detection Skull Dura Cortex

7 Signal Detection Scalp Skull Dura Cortex

8 Signal Detection EEG Scalp Skull Dura Cortex

9 Signal Detection EEG Scalp Skull Non-invasive Dura Cortex

10 Signal Detection Single Units EEG Scalp Skull Non-invasive Dura Cortex

11 Signal Detection Single Units EEG Scalp Skull Non-invasive Dura Cortex Highly invasive

12 Signal Detection Single Units ECoG EEG Scalp Skull Non-invasive Dura Cortex Highly invasive

13 Signal Detection Single Units ECoG EEG Scalp Skull Dura Non-invasive Less invasive Cortex Highly invasive

14 Single Units Systems

15 Single Units Systems

16 Single Unit Systems

17 Single Unit Systems Single Units

18 Single Unit Systems Single Units

19 Single Unit Systems

20 Single Unit Systems

21 Single Unit Systems

22 Single Unit Systems

23 Single Unit Systems Action Potentials

24 Single Unit Systems Action Potentials

25 Single Unit Systems

26 Single Unit Systems

27 Single Unit Systems Rate of firing encodes information - morris code

28 Decoding Movements Single Motor Neurons Firing Rate Georgopoulos, J Neurophys, 1982 Direction

29 Decoding Movements Single Motor Neurons Firing Rate Georgopoulos, J Neurophys, 1982 Direction

30 Decoding Movements Single Motor Neurons Firing Rate Georgopoulos, J Neurophys, 1982 Direction

31 Courtesy of Dan Moran

32 Courtesy of Dan Moran

33

34 Currently Lack Durable Effect Micromotion of implant Neuronal process retracting Schwartz, Neuron, 2006 Chronic inflammation and formation of gliotic sheath

35 Surface Cortical Physiology Don t loose the cortex for the neurons

36 Superimposed Physiologies EEG/ECoG

37 Superimposed Physiologies EEG/ECoG

38 Superimposed Physiologies EEG/ECoG

39 Superimposed Physiologies EEG/ECoG

40 = Superimposed Physiologies EEG/ECoG

41 = Superimposed Physiologies EEG/ECoG

42 = Superimposed Physiologies + EEG/ECoG

43 = Superimposed Physiologies + EEG/ECoG

44 = Superimposed Physiologies + EEG/ECoG +

45 = Superimposed Physiologies + EEG/ECoG +

46 = Superimposed Physiologies + + EEG/ECoG +

47 = Superimposed Physiologies + + EEG/ECoG +

48 = Superimposed Physiologies + + EEG/ECoG + +

49 = Superimposed Physiologies + + EEG/ECoG + +

50 Sources Pfurtscheller, e al. Clin Neurophysiol, 2003 Lopes da Silva, et al Brain Res, Manning, et al J Neurosci, Ray et al J Neurosci, Miller, et al J Neurosci, Heldman et al, IEEE, 2006.

51 Lower Frequency Bands (< 30 Hz) Θ, α, µ, β Sources Pfurtscheller, e al. Clin Neurophysiol, 2003 Lopes da Silva, et al Brain Res, Manning, et al J Neurosci, Ray et al J Neurosci, Miller, et al J Neurosci, Heldman et al, IEEE, 2006.

52 Sources Lower Frequency Bands (< 30 Hz) Θ, α, µ, β Higher Frequency Bands ( >30 Hz) γ Pfurtscheller, e al. Clin Neurophysiol, 2003 Lopes da Silva, et al Brain Res, Manning, et al J Neurosci, Ray et al J Neurosci, Miller, et al J Neurosci, Heldman et al, IEEE, 2006.

53 Anatomic Distribution of Frequencies Leuthardt et al, Neurosurgery, 2006 EEG ECoG

54 Anatomic Distribution of Frequencies Leuthardt et al, Neurosurgery, 2006 EEG ECoG

55 Closing the Loop Signal Acquisition Signal Processing User Application Power Control Band Frequency Raw ECOG User Screen Condition 1. Imagine movement, directs cursor upwards Condition 2. Rest, directs cursor downward

56 EEG Courtesy of Justin Williams

57 EEG Courtesy of Justin Williams

58 EEG: Problems with External Noise 9 Hz 17 Hz

59 ECoG ECoG Electrocorticography and the importance of gamma bands

60 Brain Computer Interfaces EEG Single Units

61 Brain Computer Interfaces EEG Single Units

62 Brain Computer Interfaces EEG Single Units Limited Degrees of Freedom Erratic function Prolonged User Training

63 Brain Computer Interfaces EEG Single Units Limited Degrees of Freedom Erratic function Prolonged User Training Lacks durable effect due to scar formation around the leads

64 Brain Computer Interfaces EEG Single Units

65 Brain Computer Interfaces EEG Electrocorticography (ECOG) Single Units ECOG signal (mv) >> EEG (µv) Frequency range >40Hz Better regional discrimination than EEG (mm vs. cm) Scarring less of an issue than single units Clinical Application

66 Current Research Model Utilizes patient with intractable epilepsy Monitored for 1 week Electrode spacing coarse 1 cm electrode spacing Electrode size 2.3 mm in diameter Key Advantages: 1. Unique Access to Human Cortex 2. Unique Access to Human Specific Cognitive Operations 1 Week

67 Current Research Model Utilizes patient with intractable epilepsy Monitored for 1 week Electrode spacing coarse 1 cm electrode spacing Electrode size 2.3 mm in diameter Key Advantages: 1. Unique Access to Human Cortex 2. Unique Access to Human Specific Cognitive Operations 1 Week

68 Current Research Model Utilizes patient with intractable epilepsy Monitored for 1 week Electrode spacing coarse 1 cm electrode spacing Electrode size 2.3 mm in diameter Key Advantages: 1. Unique Access to Human Cortex 2. Unique Access to Human Specific Cognitive Operations Test 1 Week

69 Current Research Model Utilizes patient with intractable epilepsy Monitored for 1 week Electrode spacing coarse 1 cm electrode spacing Electrode size 2.3 mm in diameter Key Advantages: 1. Unique Access to Human Cortex 2. Unique Access to Human Specific Cognitive Operations Test 1 Week

70 Current Research Model Utilizes patient with intractable epilepsy Monitored for 1 week Electrode spacing coarse 1 cm electrode spacing Electrode size 2.3 mm in diameter Key Advantages: 1. Unique Access to Human Cortex 2. Unique Access to Human Specific Cognitive Operations Test 1 Week

71 Current Research Model Utilizes patient with intractable epilepsy Monitored for 1 week Electrode spacing coarse 1 cm electrode spacing Electrode size 2.3 mm in diameter Key Advantages: 1. Unique Access to Human Cortex 2. Unique Access to Human Specific Cognitive Operations Test 1 Week

72 Higher Frequencies Have Higher Anatomic Resolution Schalk, Leuthardt et al, J Neural Engineering, 2008

73 ECoG Used for BCI Control 1 D Control Task 2 D Control Task Leuthardt et al. J, Neural Engineering, 2004 Schalk et al J. Neural Engineering, 2008

74 ECoG Used for BCI Control 1 D Control Task 2 D Control Task Leuthardt et al. J, Neural Engineering, 2004 Schalk et al J. Neural Engineering, 2008

75 ECoG

76 ECoG

77 ECoG

78 ECoG

79 ECoG

80 ECoG

81 ECoG

82 Decoding Speech

83 Continuous versus Discrete Device Control Hello, My name is Eric. Mouse Keyboard

84 Layers of Cortical Activity

85 Decoding Content

86 Decoding Content

87 Decoding Content

88

89 Phoneme Separation (oo vs ah).3 Electrode 5 mm. r Frequency Leuthardt, et al J. Neural Engineering, 2011

90 Phoneme Separation (oo vs ah).3 Electrode 5 mm. r Frequency Leuthardt, et al J. Neural Engineering, 2011

91 Envisioned Clinical Protocol Schalk and Leuthardt, in preparation

92 Envisioned Clinical Protocol Schalk and Leuthardt, in preparation

93 Envisioned Clinical Protocol Schalk and Leuthardt, in preparation

94 Envisioned Clinical Protocol Schalk and Leuthardt, in preparation

95 Envisioned Clinical Protocol Schalk and Leuthardt, in preparation

96 Wide Range of Utility Schalk and Leuthardt, in preparation

97 Seven quarters later they were having extended volleys, and the constant pong noise was a9rac:ng the curiosity of others at the bar. Before closing, everybody in the bar had played the game. The next day people were lined up outside Andy Capp's at 10 A.M. to play Pong. Around ten o'clock that night, the game suddenly died." (the machine coin container was full) VIDEO GAMES

98 Seven quarters later they were having extended volleys, and the constant pong noise was a9rac:ng the curiosity of others at the bar. Before closing, everybody in the bar had played the game. The next day people were lined up outside Andy Capp's at 10 A.M. to play Pong. Around ten o'clock that night, the game suddenly died." (the machine coin container was full) VIDEO GAMES

99 BRAIN COMPUTER INTERFACES 2006

100 BRAIN COMPUTER INTERFACES 2006

101 BRAIN COMPUTER INTERFACES 2006

102 THANK YOU! Washington University in St Louis Department of Neurological Surgery Ralph Dacey MD Josh Dowling MD Matt Smyth MD David Limbrick MD Robert Grubb, MD Thomas Woolsey, PhD Department of Biomedical Engineering Frank Yin MD, PhD Dan Moran Ph.D Dennis Barbour Ph.D. Students: Kim Wisneski, Zac Freudenburg, Mohit Sharma, Charlie Gaona, David Bundy, Jarod Roland, Jonathan Breshears, Nick Szrama, Carl Hacker, Amy Daitch Department of Neurology Maurizio Corbetta, MD Ed Hogan MD Larry Eisenman MD, PhD Samiya Rashid Department of Computer Science William Smart, PhD Robert Pless, PhD Killian Weinberger, PhD Department of Anesthesia Alex Evers, MD Rene Tempelhoff, MD Wadsworth Center, NY Gerwin Schalk PhD National Academy of Engineering Google Funding: James S. McDonnell - Higher Brain Function Grant Department of Defense: W911NF & W911NF National Institute of Health: NINDS NIH R01- EB Children s Discovery Institute Department of Radiology Avi Snyder, MD PhD Josh Shimony, MD, PhD Marcus Raichle, MD