Control principles in upper-limb prostheses electromyographic (EMG) signals generated by muscle contractions electroneurographic (ENG) signals interface with the peripheral nervous system (PNS) interface with the central nervous system (CNS) electrocorticography (ECoG) signals electrooculography (EOG ) helps in controlling prosthesis by decoding eye movement external sensory inputs from switches, foot pressure sensors, cameras, inertial measurement units (IMU), etc
electromyography detects the electrical potential generated by muscle cells signals from voluntarily contracted muscles EMG signals are the current generated by the ionic flow across membranes of the muscle fibers muscle fibers are in groups called motor units (MU) the activation of MU creates a motor unit action potential (MUAP) EMG signal is the summation of these MUAPTs (motor unit action potential trains) STEPS FOR DEVICE CONTROL signal acquisition decoding control Electromyographic EMG
S EMG non-invasive method surface electrodes are used measured potentials on skin surface Electromyographic EMG surface electrodes are able to provide only a limited assessment of the muscle activity I EMG invasive detected using needle electrodes or wires potentials of selected muscles are measured
Electromyographic EMG
Electromyographic EMG
Electromyographic EMG
Hybrid myoelectric control systems EMG+voice https://www.youtube.com/watch?v=t0fqm57bbg8 EMG+IMU https://www.youtube.com/watch?v=fmsrt832np4 EMG+manual switch https://www.youtube.com/watch?v=7qr_2n5y9pw
EMG drawbacks loss of muscles makes EMG unavailable and affect controllability main concern: sensory feedback to prevent object slippage electrocutaneous feedback can cause pain to the wearer pressure feedback Targeted Muscle Reinnervation TMR provides cutaneous sensory feedback to the amputee by relocating residual nerves to nonfunctional muscles https://www.youtube.com/watch?v=v4uqu4392wm
EMG vs EEG electromyographic (EMG) has allowed for an increase in the degrees of freedom (DOFs) of hand designs larger number of available grip patterns, little added complexity for the wearer little sensory feedback non-natural control (must be learned by the user) electroneurographic (ENG) signals more invasive than using surface EMG for control more natural control both efferent and afferent sensory feedback
Sensory feedback in bidirectional hand prostheses poor sensory feedback available to the user while grasping natural sensory information stimulating the sensory peripheral ulnar nerves real-time control of a dexterous prosthesis blindly identify compliance and shape S. Raspopovic et al. 2014 https://www.youtube.com/watch?v=61is3-l4ezi
Electroneurographic ENG electroneurographic (ENG) signals interface with the peripheral nervous system (PNS) interface with the central nervous system (CNS)
Electroneurographic ENG PNS an electrode acting as an interface with the PNS can be extraneural or intraneural most common extraneural cuff electrode, which surrounds the nerve fascicle and acquires signals from its exterior intraneural electrodes are placed directly within the tissue of the nerve
Electroneurographic ENG CNS the most invasive method of ENG control is to implant electrodes into the motor cortex of an amputated patient inside the skull controlled by microelectrodes implanted within the brain (CNS: brain and spinal cord) CNS
Neuroscience and Robotics results coming form neuroscience research: the behavior of the human hand during grasp is dominated by movements in a continuous configuration space with limited dimensions tendon couplings and muscle activation patterns exhibited by humans lead to significant joint coupling and inter-finger coordination, i.e. lead to significant movements coordination, called postural synergies
How we can close the gap? postural synergies hold great potential for robot hand s control Evidence of simplified control schemes at neurological level for the organization of hand movements The first two synergies account for >80% of the hand configurations variance reduction of the grasp synthesis problem dimention control of the robotic hand in a space of highly reduced dimensions with respect to the number of DOFs prosthetic devices provided of simple interface and control strategies based on few EMG input
Postural synergies for human-like grasping the DEXMART HAND and UB Hand IV are innovative robotic hands with the same mechanical design (different thumb kinematics) the hand design aims to the maximum simplification endoskeletal structures pin joints integrated into the phalanges manufacturing and assembly complexity have been reduced by systematic parts integration adopting fusion deposition manufacturing
Hand Modelling the mechanical structure remotely located actuators with tendon-based transmissions routed through fixed paths (sliding tendons), N+1 tendon configuration surface compliance is introduced through a purposely designed soft cover mimicking human skin the kinematics structure the DEXMART Hand presents a total amount of 20 DOFs, the medial and the distal joints are coupled by means of an internal tendon thumb with different kinematics and joint limits w. r. t. the other fingers, humanlike manipulation capabilities and mobility (opposition with the other four fingers) UB Hand IV DEXMART Hand
Human Hand Observation different methods for the observation and different tecnologies (cyber gloves, motion tracking systems ) human fingertip position are detected using low-cost RGBD camera (Kinect) human grasps are mapped directly on the robotic hand kinematics by means of CLIK the kinematics is suitably scaled on the basis of the human hand dimension
Mapping the human hand grasps five subjects with different hand size have been involved in the experiments to determine the hand pose with respect to the camera for the i-th subject fingertips and panel reference points are detected in the open hand configuration the affine transformation between the camera and the hand reference frame is obtained the position of the panel reference points with respect to the hand frame is computed
Mapping the human hand grasps each subject performs the set of 36 postures represented in the table the mapping from the human hand fingertip positions and the robotic hand joint positions is performed: by scaling the robot hand link dimensions to fit with the human subject by inverting the robot hand kinematics by means of CLIK
The three predominant synergies UB Hand IV DEXMART Hand
Synergies Computation on the Grasping Dataset configuration matrix of the reproduced grasps PCA on the grasps offset matrix base of the selected synergies (> 85%) zero offset each hand grasp can be approximated by a suitable selection of the synergy coefficients the projection of each robotic hand configuration on the postural synergies subspace is evaluated:
Grasping Control with Postural Synergies hand configuration for the i-th grasp computed as interpolation of the weight values in three configurations zero offset open hand desired grasp (object dependent)
Control with Postural Synergies Power grasps Precision grasps Intermediate grasps Synthesized grasps
Differential Kinematics Synergies Framework for Design and Control of Underactuated Artificial Hands 25/37 Differemtial mapping between Mechanical Synergies Space and Cartesian Space Mechanical Synergies Jacobian Hand Jacobian Differential mappingbetween Mechanical Synergies Space and Joint Space Fingertips Position Joint Angles Motor Variables
Control in the Synergies Subspace Synergies Framework for Design and Control of Underactuated Artificial Hands 26/37 Neuroscience Studies human beings realize grasping actions by means of first phase of pre-shaping exploiting synergies learning strategies and vision second phase of adaptation to the object that requires mainly contact forces sensory information
Control in the Synergies Subspace feedforward term for hand pre-shaping the configuration selected within the data-set used for synergies computation on the basis of the similarities related to the object shape and size, and to the grasp type, namely power, precision or lateral grasp local adaptation realized in a synergy-based framework exploiting force and position feedback grasp optimization synergy-based quality index relying on force closure property current regulation to avoid high contact forces
Experimental Results Synergies and Underactuation: Learning and Control Strategies for Anthropomorphic Hands 28/37 F. Ficuciello, A. Federico, V. Lippiello, B. Siciliano, Synergies Evaluation of the SCHUNK S5FH for Grasping Control, ARK 2016