A platform for researching on multimodal robot-assisted rehabilitation therapies

Transcription

1 The Fourth IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics Roma, Italy. June 4-7, 01 A platform for researching on multimodal robot-assisted rehabilitation therapies Ricardo Morales, Francisco Javier Badesa, J. Rodriguez, Nicolás García-Aracil José María Azorin, Carlos Pérez-Vidal Abstract This paper presents the development of a rehabilitation robotic platform to research on the adaptation of the complexity of robot assisted therapy and the real-time displays of a virtual reality system in accordance with specific patient requirements. First, a kinematic analysis of the human arm during typical daily activities is presented to justify the design requirements of the robotic rehabilitation platform. Then, the other necessary components to monitor multi-sensory data from the subject and robotic side and to stimulate the subject during the task are described. Finally, first experimental results with the platform are presented to show that the therapy can be modulated in function of the patient state. I. INTRODUCTION According to the World Health Organization, the number of people over 65 years will increase by 73 percent in the industrialized countries and by 07 percent worldwide. By 050, the percentage of the European population over 65 years should almost double from 1.3 to 0.6 percent (from 40 to 80 million). This age group is particularly prone to cerebral vascular accident (CVA), also known as stroke. The relative incidence of stroke doubles every decade for people over 55 years old. In fact, stroke is the leading cause of permanent disability in industrialized nations. Each year over 90,000 Europeans and 700,000 North Americans have a stroke; more than a half survive but often with severe impairments. The main symptoms are loss of muscle strength, spasticity and lack of coordination of muscle activation [1]. Therefore, an interdisciplinary rehabilitation program to provide integrated care for people that survive a stroke is required. The use of robotic devices, as a possible rehabilitation strategy to achieve motor recovery, can be justified because of its potential impact on better therapeutic treatment and motor learning. Researchers have demonstrated the effectiveness of repetitive grasp and release exercises [], constraint induced therapy for the paretic limb [3] [4], increased intensity or duration of therapy including external manipulation [5], bio-feedback [6], bilateral movement training [7] [8] and robot-assisted therapy [9] [10] [11] [1] in restoring motor function in the paretic upper limb during acute and chronic stages of stroke recovery. In This work has been partly supported by the European Commission (FP7- ECHORD MAAT experiment) and by the Spanish Government through the project Interpretacion de la Intenci n y Actuaci n Humana mediante Se ales Biomedicas y el Analisis Cinematico y Dinamico del Movimiento (DPI C04-04). Authors thanks J.A. Martinez-Terres for his help on the development of the electronics for the control Authors are with the Department of Systems Engineering and Automatic Control, Miguel Hernandez University, Spain r.morales@umh.es any case, the therapeutic approach is well structured and repetitive in order to promote cortical reorganization after stroke [13] [14]. Recently, a scientific statement published by the American Heart Association in the Comprehensive Overview of Nursing and Interdisciplinary Rehabilitation Care of the Stroke Patient reports the best current evidences and recommendations for interdisciplinary management of poststroke rehabilitation including robot-assisted therapy [15]. Their recommendations and levels of evidence for treatment of motor issues regarding robot-assisted therapy are Class I, Level of Evidence A for stroke care in the outpatient and chronic care settings and Class IIa, Level of Evidence A for stroke care in the inpatient settings. It is well-known that classical robot-aided rehabilitation therapy consists of a robot guiding linked with a virtual reality system that motivates the patient to accomplish a predetermined motion. On the other hand, a new trend in the development of novel rehabilitation systems is placing the patient in the control loop [16] [17] [18] [19] [0]. Furthermore, the main achievements of MIMICS research project funded by the European Commission were in the line of the development of biocooperative control systems for adapting the robotic therapy in order to maximize patient motivation. The paper is structured in two sections: Section II presents the description of the design requirements of the multimodal robotic platform along with the final robotic system developed to fulfill these requirements. Moreover, the systems for measuring the multi-sensory data (such as pulse, skin conductance, skin temperature, position, velocity, etc.) are described too. First experimental results with this platform are reported in Section III to show that it is possible to adaptively and dynamically change complexity of the therapy and real-time displays of a virtual reality system in accordance with specific patient requirements. II. MULTIMODAL ROBOTIC PLATFORM The multimodal robotic platform has been developed to assess the benefits that robot-assisted rehabilitation of upper extremities for Activities of Daily Living (ADL), including patient in the loop, could produce. The system will be able to record multi-sensory data (such as motion, forces, voice, muscle activity, heart rate, skin conductance etc.) and process them for a continuous estimation of the patients intentions and physiological state. These information will be used to (i) online update robot control during the execution of an assisted ADL task in order to guide, help or force the patient /1/$ IEEE 1398

2 z a x y b c Fig. 1. Simplified kinematic model of human arm proposed by [1] limb toward the target in accordance to his/her residual motion capabilities, (ii) apply corrective actions in case of incorrect motion; (iii) change the immersive virtual reality system in order to increase patient active involvement and maximising his/her motivation. The platform is composed by: a robotic arm for robot therapy administration; a force sensor for providing force feedback to robot control; a set of wearable sensors/actuators for motion analysis and stimulation;and a physiological parameter recording system. II-A. Robotic arm The robotic arm has been designed using the information provided by a previous analysis of activities of daily living (ADL) and a simple model of human arm reachable workspace (ARW) [1]. The model of the human ARW includes three rigid links, the shoulder link representing the clavicula and scapula, the upper arm link representing the humerus, and the forearm link representing the radius and ulna. This model does not describe the motion of singular bones, but the spatial motion of the reference point on the wrist which is enough for the definition of the robotic workspace in our case (Figure 1). Because of the anatomical properties of the arm, bones, and muscles, the lower and the upper limits of joint angles depend on the values of other joint angles. In the case of right human arm, the limits of the flexion-extension q 4 are specified in terms of linear functions of the abduction q 3 and the limits of the rotation are given as functions of the values of the angles of abduction-adduction q 4 and flexionextension q 3. The limits on other joint angles are considered constant and independent. Their values and the lengths of the Fig.. Human Arm Workspace shoulder, upper arm and forearm links (a, b, c in Figure 1) are extracted from [1]. The human workspace reachable by the subject wrist can be obtained by computing the position of the wrist for all the joint angles of the model inside their limits. The human ARW is shown in Figure The second task was to carry out a previous analysis of the workspace needed for 5 typical activities of daily living (ADL) like eating with spoon, combing hair, drinking water with a cup, washing face and brushing teeth []. The kinematic data of the human arm during ADL tasks were collected using two wireless inertial measurement units (IMUs) attached to subject s arm and forearm. The experiments were conducted in a dedicated room at the Bioengineering Institute of Universidad Miguel Hernandez. Two people were present: the subject and the experiment supervisor. Before starting the experiment, subjects were asked to perform three repetitions of each ADL task. The ADL tasks were performed in either standing or sitting body posture depending on the nature of the activity interacting with various small objects related with the performed ADL task (Figure 3). Fifteen students and staff members (11 males and 4 females) participated in the experiment. All of them were healthy, without any major cognitive or physical deficits. They were aged between 0 and 41, mean age 8.0 years, median age 6 years, standard deviation 6.6 years. For each arm motion, the rotation matrix of each IMU is computed. Using the simplified kinematic model of human arm and the information provided by the IMUs, trajectories for each ADL task in the worst case can be computed. In Figure 4, the human ARW with the trajectories for each ADL task in the worst case are shown. It can be seen that the ADL tasks are inside the computed human ARW and a sub- 1399

3 seen that the robot workspace is large enough to reach all the position of the wrist during typical ADL tasks. II-B. Force sensor The robot was designed to mount a ATI force/torque sensor which is used to measure the interaction forces and torques between the user and the robot and a electromagnetic system which is used to release the subject s arm in case of a dangerous situation. II-C. Wearable sensor/actuators A set of wearable sensors/actuators for motion analysis and stimulation are used to compute the position of the subject s arm and to stimulate the subject. Motion capture system consits of a wireless IMUs based on commercial electronic boards and a vibrotactile stimulation system based on a wireless commercial electronic board with different vibro motors. Fig. 3. IMUs mounted on one subject s arm and foream workspace for ADL tasks inside the human ARW can be identified. Based on the results of the analysis of the ADL tasks and human ARW, a reduced human ARW of rehabilitation robot for ADL tasks were generated. The robot was designed to reach the workspace necessary for performing ADL tasks in either standing or sitting body posture. Finally, we defined a 7 DOF robot configuration using PRL modules manufactured by Schunk and custom-made links. The designed links along with the Schunk modules creates a kinematic chain. A world coordinate frame is established in the base of the robot. In order to determine the transformation matrix from the world frame to the end-effector frame, a local coordinate frame needs to be established for every link of the robot. Each local coordinate frame is defined using Denavit-Hartenberg (D-H) convention. The figure 5 shows the coordinate frames of the robot and the distance between them. The relations between the local frames can be expressed with a D-H table. Table II-A shows the robots parameters expressed in meters and radians. In Figure 6, the robot workspace along with II-D. Physiological parameter recording system Physiological signals (pulse, skin conductance, skin temperature and respiration rate) are sampled at.4 khz using a g.us-bamp signal amplifier from g.tec Medical Engineering GmbH. Pulse was measured using a g.pulsesensor. The plethysmographic pulse sensor is attached to the distal phalanx of the thumb. Skin conductance was measured using a g.gsr sensor manufactured by g.tec. The electrodes were placed on the medial phalanxes of the second and third fingers of the unaffected hand. The sensor generated a constant voltage between the two electrodes and measured the current between the electrodes in order to estimate skin conductance. Skin temperature was measured using a g.tempsens sensor. The thermistor was placed on the palmar surface of the Link θ i d i a i α i π π π π π π π TABLE I DENAVIT HARTENBERG PARAMETERS the reduced workspace for ADL tasks are shown. It can bee Fig. 4. Human Arm Workspace with ADL trajectories 1400

4 Y w' Yw' w' 0 0 Y 0 Y1 1 1 Y3 4 4 Y 4 Y5 6 6 Y Y 7 7 Fig. 7. End-effector and force sensor w w Y w Fig. 5. Robot Kinematics analysis. Fig. 6. RobotWorkspace with ADL de pie distal phalanx of the fifth finger. The respiration rate was measured using a g.flowsensor. The respiration rate sensor is a thermistor sensor placed beneath the nose, with the larger part covering the mouth and the smaller part bent slightly so it does not enter the nose. All signals were acquired and processed directly in Matlab and Simulink (The Mathworks, Natick, MA). III. EPERIMENTAL RESULTS To evaluate the multimodal robot-assisted rehabilitation platform, s virtual reality software has been developed with adjustable level of task difficulty which can be adjusted as a function of the evolution of the subject. The developed software creates a virtual world for activities of daily living, such as taking a glass, drinking and placing object on shelves, etc. and the robot creates the interaction between the real world and the virtual world through simulation of the interaction forces between patient s virtual arm and objects to be manipulated. The basic scenario consists of a table inside a virtual kitchen, a glass and a coaster. At the beginning of the task, the glass appears on a random position over the table, the subject must grasp the glass. When the glass has been grasped, a coaster appears on a random position over the table and the subject must leave the glass over the coaster. The subject has a limited time to carry out the task and it is displayed in the screen through a progress bar. Moreover, the multimodal virtual rehabilitation system has been designed to adjust three modalities of multimodal feedback: visual, acoustic and haptic. The estimation of subject s physiological state, motor and virtual task performance will produce the changes of visual, acoustic and haptic feedback reflected to the subject with the objective of maximizing the motivation of the subject and improving the therapeutic outcomes. The implemented multimodal feedback modalities are the following: 1. Haptic actions: i) basic force feedback for collision with the environment without assistance or resistance, ii) trajectory guidance through force fields, iii) damping force field related to the approaching velocity to the target and iv) force field for disturbances v) resistance force field related to the position error between the coaster and the current position of the glass.. Acoustic actions: i) relaxing sound played as music, ii) motivating sounds when subject successfully grasps 1401

5 the glass and/or leaves the glass over the coaster and iii) encouraging statements to additionally motivate the subject 3. Visual actions: i) basic game scenario without environment, ii) home scenario representing a virtual kitchen and iii) animations to motivate the subject The combinations of these multimodal feedbacks allow us to define different task levels to be selected according to the estimation of the subject s progress during the virtual rehabilitation task: 1. Level 1: The haptic action is a basic force feedback for collision with the environment without resistance and assistance, the acoustic action is a relaxing sound played as music and the visual action is a basic game scenario without environment.. Level : The haptic action is a resistance force field related to the approaching velocity to the target, the acoustic action consist of a motivating sounds when subject successfully grasps the glass and/or leaves the glass over the coaster and the visual action implemented is a home scenario representing a virtual kitchen. 3. Level 3: The haptic action is a force field for disturbances, the acoustic action consist of a motivating sounds when subject successfully grasps the glass and/or leaves the glass over the coaster and the visual action is a home scenario representing a kitchen and animations to motivate the subject. 4. Level 4: The haptic action is resistance force field related with the position error between the coaster and the current position of the glass, the acoustic action consist of a motivating sounds when subject successfully grasps the glass and/or leaves the glass over the coaster and encouraging statements to additionally motivate the subject and the visual action is a home scenario representing a virtual kitchen. illustrate the performance of the proposed multimodal robotassisted rehabilitation system, the level changes during the execution of a virtual rehabilitation task by a typical subject shown in Figure 8. The estimation of physiological and motor state and the evaluation of the required level changes is carried out every 150 seconds and it is represented as a red vertical line in Figure 8. We used pulse and skin conductance level(scl) as physiological feedback and their relations with the level changes are shown in Figure 8. The level changes depend of the estimation of arousal and valence in function of the changes in physiological signals (pulse and SCL). We can see in the figure that the arousal and valence increase or decrease with the increasing/decreasing of pulse and skin conductivity level according to the fuzzy rules implemented. IV. CONCLUSION Our platform for researching on multimodal robot-assisted rehabilitation therapies has been presented and described. Moreover, experimental results with this platform have been reported to show that a combination of pulse and skin conductance level seems to be a robust method of physical and cognitive workload estimation in virtual rehabilitation tasks assisted by a robotic device and It is possible to modulate the complexity of the virtual rehabilitation task using artificial intelligent techniques in order to estimate subject s performance and physiological state. From the acquired experience in this research, it seems that the use of adaptive algorithms for intelligent machine learning could be the basis for future rehabilitation devices that automatically adapt the delivered therapy to the specific needs and demands of the patient. In general terms, our approach based on fuzzy logic systems works fine but it still needs extensive evaluation. Moreover, the evaluation and comparison of other artificial intelligent techniques would be in line with our next developments inside this topic. Pulse rate (rpm) GSR (us) Pulse Rate Time (s) Skin conductance level (SCL) Time (s) Fig. 8. Changes in rehabilitation task level in function of changes in pulse and skin conductivity level To modify the complexity of the therapy, artificial intelligence techniques can be applied. In short, we have selected fuzzy logic systems for their simplicity and because they don t need large training sets. The final solution implemented in this experiment is based on a fuzzy logic system in two hierarchical levels: first, the estimation of subject s physiological and motor state using the physiological signals and the measurements of motor performance and second, the estimation of level changes using the provided information about the estimation of physiological and motor state [3]. To LEVEL 4 LEVEL 3 LEVEL LEVEL 1 LEVEL 4 LEVEL 3 LEVEL LEVEL 1 REFERENCES [1] RPS van Peppen, G. Kwakkel, BC Harmeling van der Wel, BJ Kollen, JSM Hobbelen, JH Buurke, JHC Halfens, L Wagenborg, MJ Vogel, M Berns, R van Klaveren, HJM Hendriks, and J Dekker. Kngf clinical practice guideline for physical therapy in patients with stroke. review of the evidence. Nederlands Tijdschrift voor Fysiotherapie, 114(5), 004 [English Translation 008]. [] C Buetefisch, H Hummelsheim, P Denzel, and K-H Mauritz. Repetitive training of isolated movement improves the outcome of motor rehabilitation of the centrally paretic hand. Journal of the Neurological Sciences, 130:59 68, [3] E. Taub, N. E. Miller, T. A. Novack, E. W. Cook, W. C. Fleming, C. S. Nepomuceno, J. S. Connell, and J. E. Crago. Technique to improve chronic motor deficit after stroke. Arch. Phys. Med. Rehabil., 74: , [4] J. H. van der Lee, R. C. Wagenaar, G. J. Lankhorst, T. W. Vogelaar, W. L. Deville, and L. M. Bouter. Forced use of the upper extremity in chronic stroke patients: results from a single-blind randomized clinical trial. Stroke, 30: , [5] M Dam, P Tonin, S Casson, M Ermani, G Pizzolato, V Iaia, and L Battistin. The effects of long-term rehabilitation therapy on poststroke hemiplegic patients. Stroke, 4: , [6] S.L. Wolf. Electromyographic biofeedback applications to stroke patients. a critical review. physical therapy. Physical Therapy, 63: ,

6 [7] T.M. Matyas and M.H. Mudie. Can simultaneous bilateral movement involve the undamaged hemisphere in reconstruction of neural networks damaged by stroke? Journal of Disability and Rehabilitation, (1/):3 37, 000. [8] P. S. Lum, C. G. Burgar, P. C. Shor, M. Majmundar, and M. Van der Loos. Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper-limb motor function after stroke. Arch Phys Med Rehabil, 83:95 959, 00. [9] M. L. Aisen, H. I. Krebs, N. Hogan, F. McDowell, and B. T. Volpe. The effect of robot-assisted therapy and rehabilitative training on motor recovery following stroke. Arch Neurol, 54: , [10] H. I. Krebs, N. Hogan, M. L. Aisen, and B. T. Volpe. Robot-aided neurorihabilitation. IEEE Trans. Rehabil. Eng, 6:75 87, [11] B. T. Volpe, H. I. Krebs, N. Hogan, L. Edelstein, C. Diels, and M. L. Aisen. Robot training enhanced motor outcome in patients with stroke maintained over three years. Neurology, 53( ), [1] S. D. Fasoli, H. I. Krebs, J. Stein, W. R. Frontera, and N Hogan. Effects of robotic therapy on motor impairment and recovery in chronic stroke, archives of physical medicine and rehabilitation. Archives of Physical Medicine and Rehabilitation, 84:477 48, 003. [13] R. J. Nudo and K. M. Friel. Cortical plasticity after stroke: implications for rehabilitation. Rev Neurol, 155: , [14] W. R. Staines, W. E. McIlroy, S. J. Graham, and S. E. Black. Bilateral movement enhances ipsilesional cortical activity in acute stroke: A pilot functional mri study. Neurology, 51: , 001. [15] Elaine L. Miller, Laura Murray, Lorie Richards, Richard D. orowitz, Tamilyn Bakas, Patricia Clark, Sandra A. Billinger, on behalf of the American Heart Association Council on Cardiovascular Nursing, and the Stroke Council. Comprehensive overview of nursing and interdisciplinary rehabilitation care of the stroke patient: A scientific statement from the american heart association. Stroke, 41(10):40 448, 010. [16] A. Koenig, D. Novak,. Omlin, M. Pulfer, E. Perreault, L. immerli, M. Mihelj, and R. Riener. Real-time closed-loop control of cognitive load in neurological patients during robot-assisted gait training. Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 19(4): , 011. [17] Alexander Duschau-Wicke, Andrea Caprez, and Robert Riener. Patient-cooperative control increases active participation of individuals with sci during robot-aided gait training. Journal of NeuroEngineering and Rehabilitation, 7(43), 010. [18] R. Riener and M. Munih. Guest editorial special section on rehabilitation via bio-cooperative control. Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 18(4): , 010. [19] D. Novak, M. Mihelj, J. iherl, A. Olensek, and M. Munih. Psychophysiological measurements in a biocooperative feedback loop for upper extremity rehabilitation. Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 19(4): , aug [0] C. Rodriguez Guerrero, J. Fraile Marinero, J. Perez Turiel, and P. Rivera Farina. Bio cooperative robotic platform for motor function recovery of the upper limb after stroke. In Engineering in Medicine and Biology Society (EMBC), 010 Annual International Conference of the IEEE, pages , sept [1] J. Lenarcic and A. Umek. Simple model of human arm reachable workspace. Systems, Man and Cybernetics, IEEE Transactions on, 4(8): , aug [] J. Rosen, J.C. Perry, N. Manning, S. Burns, and B. Hannaford. The human arm kinematics and dynamics during daily activities - toward a 7 dof upper limb powered exoskeleton. In Advanced Robotics, 005. ICAR 05. Proceedings., 1th International Conference on, pages , july 005. [3] M. Mihelj, D. Novak, and M. Munih. Emotion-aware system for upper extremity rehabilitation. In Virtual Rehabilitation International Conference, 009, pages , july