Evaluation of a Stretch Sensor for its Inedited Application in Tracking Hand Finger Movements

Transcription

1 This full text paper was peer-reviewed at the direction of IEEE Instrumentation and Measurement Society prior to the acceptance and publication. Evaluation of a Stretch Sensor for its Inedited Application in Tracking Hand Finger Movements Laura Sbernini, Antonio Pallotti, Giovanni Saggio Department of Electronic Engineering University of Rome Tor Vergata Via del Politecnico 1, Rome, Italy saggio@uniroma2.it Abstract Flex sensors are frequently used as wearable tools for unobtrusively tracking human joint movements. Among all the flex sensor types, the resistive ones are the mostly adopted thanks to their electrical and mechanical properties capable to furnish electrical resistance values related to the amount of mechanical flexion. In particular, resistive flex sensors have been finding many applications when embedded into gloves, in order to evaluate fine flexion/extension movements of the finger joints. Within this frame, here we investigate the possible utilization of a different type of flex sensors embedded into gloves, i.e., the stretch ones, since the stretch sensors change in resistance proportionally to their stretch that can be just obtained when laid on-top of a finger joint. In such a view, here we compare the characteristics of commercial flex and stretch sensors obtained by means of an ad-hoc setup and protocol. Results demonstrate the different peculiarities of the two different types of sensors, so to determine when it is convenient to adopt one type instead of the other. Keywords flex sensor; stretch sensor; sensor characterization; hand finger tracking. I. INTRODUCTION In the recent years, wearable devices able to track human motion have been proposed in a wide range of applications related to the medical field, such as: evaluating patients mobility during rehabilitation or after a clinical intervention; assessing motor disorders following a neurological disease; monitoring daily home activities; objectively assessing surgeons technical skill [1, 2]. The mentioned applications require devices to be accurate, feasible, low-cost and easy to use. The currently gold-standard technologies are mainly based on optoelectronic devices that, although being accurate and feasible, are expensive, complex to use and need a dedicated environment and a clear line of sight between body-worn markers and cameras. These limitations are overcome by using wearable devices such as resistive flex sensors (RFSs), which have been easily adopted in measuring different body parts in upper and lower limbs, in head and torso [3-6]. A RFS consists in an ink-based cracked conductive carbon material in a binder that, when flexed, separates the cracks so to increase the electrical resistance. In such a manner, this type of sensor converts the bending of the matched body part to a variation of the electrical resistance value. Thanks to their small dimensions, low cost and ease of use, RFSs are suitable to record unobtrusively and reliably human movements over a continuous period of time [6]. One of the main utilization of the RFSs is the equipment of sensory gloves useful to track fine movements of each finger joints [5-7]. Small dimensions and lightness of RFSs make it possible to realize a comfortable sensory glove, which does not obstruct hand movements in any way. Accordingly, RFSs have been proposed as a valid alternative to other type of devices investigated for the same purposes, such as optical fibers [8], inertial units [9], and Hall effectbased sensors [10]. Another fundamental aspect of RFSs is that they can be placed upon the dorsal aspect of fingers and track even the motion of each joint of a finger (metacarpophalangeal, proximal interphalangeal and distal interphalangeal joints) without overlapping themselves. With the aim to investigate a new alternative to RFSs for sensory glove equipment, in this study we evaluated a lowcost and commercially available resistive stretch sensor (SS). A SS consists commonly in an elastic and conductive rubber, which changes the electrical resistance when stretched. In order to measure human movements, a SS can be placed lengthwise across a joint, such as the knee [11], the wrist [12] or the finger one [13]: as the joint bends, the sensor is stretched causing a change of the resistance value proportionally to the angular displacement of the joint. Despite the potentiality of stretchable sensors in measuring joint motion, SSs have not been commonly applied in developing sensory gloves till now. The SSs investigated in this work consist of a cylindrical body cord, 0.06 inches in diameter, made of a polymeric component. They have been used for tracking human joint movements in a recent work devoted to the tracking of human knee [14] but, as far as we know, they have never been applied in a sensory glove before. In addition, we evaluated and compared the mostly used and commercially available RFSs together with the selected SSs. The analysis is made in terms of the curve linking the electrical resistance variation with the angular displacements, focusing on the degree of linearity, on the standard deviation and on the (eventual) hysteresis. This paper was partially based on a work supported by the Italian Space Agency (ASI), contract # R /16/$ IEEE

2 These sensors can be usefully characterized by means of a mechanical setup, which mimic the flexion/extension of a joint. For example, in [15] and [16] a dummy finger is adopted to bend flex sensors respectively in the range and To evaluate knee flexion/extension, in [17] a custom-made sensor is attached to a flexible substrate with one end fixed and one end free to pivot around. The structure was flexed in the range 0-90 and the angular displacement was evaluated in comparison to a commercial electrogoniometer response. In this study, we used a completely automatic setup and a standard protocol suitable to characterize the sensors under test. Both RFSs and SSs were characterized and their output responses were compared. (a) II. MATERIALS A. Sensors Both characterized RFSs and SSs are commercially available, respectively by Flexpoint (Flexpoint Sensor Systems, Inc., Draper, UT, USA) and Images (Images SI, Inc., Staten Island, NY, USA). Flexpoint RFSs are commercialized in three sizes (1, 2 and 3 ) and with or without a protective coating, which can be a polyimide or a polyester one (see Fig. 1a). For our purposes, we limited our attention to the 2 size, as the better length to match one finger joint for the design of a sensory glove, in order to avoid overlapping two sensors on the same finger. Images SSs come with a nominal resistance of 1000 per linear inch, with the recommendation from the manufacturer to do not exceed a 50% of elongation, so to not overcome the elastic limit. These flexible stretch sensors are commercialized in different lengths from 2 to 14 ; the length of 2 was believed as appropriate for the purpose of this study (see Fig. 1b). Both types of sensors were evaluated in measuring angular displacements actuated with a fully automatized and motorized setup, so to overcome human errors. B. Experimental setup A custom-made automatic setup was assembled in order to characterize the bend sensors mimicking the flexion/extension movement of a finger joint (see Fig. 2). To this aim, a sample of the sensors was fixed to a hinge with a knuckle of 1.2cm in diameter. Two notched leaves extended outward from the knuckle that rotated around a pin, so to simulate the rotation of the joint between two adjacent phalanges. A leaf was fixed while the other one can move thanks to the rotation of a stepped motor (PD TMCL, by Trinamic, Hamburg, Germany). An end of each RFS under test was fixed to the unmovable leaf and the other end was free to slide over the mobile leaf. The body of the sensor was kept adherent to the hinge thanks to two transparent plastic parts as shown in Fig. 2a. In this way, the sensor was only bent and the change of resistance was not due to longitudinal elongation. In (b) Fig. 1. (a) Three flex sensors (by Flexpoint) with lengths of respectively 1 (no coating), 2 (polyimide coating) and 3 (polyester coating), wide of 0.28 and thickness of (b) Three stretch sensors (by Images) with lengths of respectively 2, 4 and 6, and a diameter of particular, the RFSs do not work bi-directionally, so that the rotations were imposed to bend outwardly the printed side of the sensor. Each SS under test was fixed to both leaves thanks to two transparent plastic parts (see Fig. 2b); accordingly, the rotation of the mobile leaf stretched lengthwise the sample changing its resistance value. III. METHODS Each tested sensor was bent between 0 and 120, stepped 10, based on the possible range of motion of a finger joint [18]. The motor was set to transmit a slow rotation (about 1 minute to achieve 120 ), in order to obtain a quasi-static characterization. After each step, the position was maintained for 6s during which the resistance values of the sensor under test were measured with a frequency of 2Hz by a 5.5 digits digital multimeter (34405A by Agilent, Santa Clara, CA, USA) and the recorded data were averaged. The same method was followed to unfold the sensor (from 120 to 0 ), the hysteresis being a feature to investigate. All the procedure was repeated 10 times for each sensor and the mean value among the repetitions was obtained for each bending position. As a result, the mean responses of both types of sensors to angular displacements were evaluated and compared. A first part of the study was aimed at choosing one out of the three different designs of the tested RFS (no-coating, polyimide-coating, polyester-coating), so that five samples

3 (a) Ten SSs were characterized too, in order to compare their electrical behavior with respect to the RFSs. The final mean range, the mean standard deviation and the mean hysteresis values were taken into account, as well as the results obtained by fitting the final mean output data with a polynomial curve of the first and of the second order respectively. The fitting results were evaluated in terms of R- square values. The setup and the measurement procedures were automatically managed by means of an ad-hoc custom-made LabVIEW (LabVIEW 2012, National Instruments, Austin, TX, USA) routine, which allowed controlling motor movements and multimeter readings at the same time. Data analysis was performed using Matlab (Matlab R2014, The MathWorks, Inc., Natick, MA, USA). IV. RESULTS AND DISCUSSION (b) Fig. 2. Placement of the sensors on the automatic setup: (a) flex sensors (45 bending position); (b) stretch sensors (flat position). To simulate the rotation of a joint between two adjacent phalanges, a leaf of the hinge (right) was fixed while the other one (left) can move thanks to the rotation of a stepped motor. for each sensor group were tested. The mean output among the five samples was analyzed in terms of range and percentage increase of the resistance (with respect to the value in the flat position), of standard deviation and of hysteresis. The two latter findings were analyzed also in terms of percentage of the mean range. According to the results of these measurements, we selected one RFS design and characterized other five sensors of that group, for a total of 10 tested RFSs. A. Flex sensors coatings comparison Table 1 shows the results of the characterization of the 15 flex sensors (5 samples for each coating design) so as to compare the behaviors of the three different coatings. RFSs polyimide-overlaminated presented the lowest mean range and the lowest percent of variation, with respect to the nominal resistance value of the sensors when un-bended. Furthermore, these sensors showed the highest hysteresis and the lower repeatability (i.e. the mean standard deviation presented the highest percentage with respect to the mean range). The mean range, conversely, represented a percent increase of the resistance value higher than 1000%, both for the sensors without coating and for the polyesteroverlaminated ones. The lowest standard deviation, in terms of percentage of the mean range, was showed by the polyester-laminated sensors; conversely, the lowest hysteresis is showed by the un-coated sensors. The polyester over-laminated sensors outperformed the no-coated and the polyimide over-laminated ones due to the best combination of sensitivity (output range), hysteresis, and repeatability (standard deviation). In addition, the coating reduces the mechanical stress of the sensors but in spite of a reduction of the sensitivity. We had to evidence that the presence of a coating brings to a decrease of the range of the sensor resistance value during the bending but, conversely, increases the stability of the response over time [16]. Results of the linear and quadratic fitting of the mean Flex sensor coating TABLE 1: COMPARISON AMONG THE THREE FLEX SENSOR DESIGNS: NO-COATING, POLYESTER COATING AND POLYIMIDE COATING. Mean resistance in flat position Mean Range (% increase with respect to the resistance in flat position) Mean std a (% of the mean range) Mean hysteresis (% of the mean range) R 2 poly1 b no coating 7.02 k k (>1000%) 3.29 k (2.15%) 0.60 k (0.39%) polyester 4.75 k k (>1000%) 0.77 k (1.15%) 0.90 k (1.35%) polyimide k k (438%) 3.13 k (4.79%) 2.15 k (3.29%) R 2 poly2 c a. std: standard deviation b. poly1: linear polynomial curve c. poly2: quadratic polynomial curve

4 output curves (R-square) are also given in Table 1. The three different RFS configurations did not present any substantial difference showing a similar behavior in terms of linearity: sensor response data were fitted better by a quadratic polynomial curve (R-square > 0.990) than by a linear polynomial curve (R-square < 0.990). As a result, we selected the polyester over-laminated RFSs as the most interesting sensors to be integrated into a sensory glove. This choice was in accordance with other works too [15, 16]. B. Flex vs. Stretch sensors The output of the 10 polyester-overlaminated RFSs is shown in Fig. 3, as the mean curve with the relative standard deviation. The mean range of the final curve is 64.01k, which represents an increase greater than 1000% with respect to the mean resistance in flat position (4.84k ). The mean standard deviation is as low as 0.96k, which represents the 1.49% of the mean range. The hysteresis between the fold and the unfold phase has a mean of 0.90k, which represents the 1.41% of the mean range. Fitting the final mean output data furnishes the results shown in Fig. 4: the R-square value relative to a linear fitting is and the R-square value relative to a quadratic fitting is The same analysis was conducted for the 10 SSs and the final output is shown in Fig. 5 as the mean curve and the relative standard deviation. The mean range of the final curve is , which represents an increase of 54% with respect to the mean resistance in flat position ( ). The mean standard deviation is 18.97, which represents the 3.77% of the mean range. The hysteresis between the fold and the unfold phase has a mean of 15.28, which represents the 3.04% of the mean range. Fitting the final mean output data furnishes the results shown in Fig. 6: the R-square value relative to a linear fitting is and the R-square value relative to a quadratic fitting is To compare the behavior of the two mean responses, both curves are shown in Fig. 7. It is important to notice that both types of sensors presented a monotone curve as response to the angular displacements in the range 0-120, both for the fold and the unfold bending. The two types of Fig. 3. Output of the 10 polyester-overlaminated flex sensors: mean and standard deviation. Fig. 5. Output of the 10 stretch sensors: mean and standard deviation. Fig. 4. Output of the 10 polyester-overlaminated flex sensors: mean, linear and quadratic fitting curve. Fig. 6. Output of the 10 stretch sensors: mean, linear and quadratic fitting curve.

5 Fig. 7. Comparison between flex sensors and stretch sensors: mean curves among 10 samples (stright curve is related to the left axis, dotted curve to the right one). sensors differ from each other in terms of linearity. RFSs behavior can be fitted by a second-order polynomial curve (R-square=0.996). In particular, it is worthwhile to notice that we can evidence two distinct sections of the curve: one for the bending angles before 30 and one for greater angles. In fact, the first part of the output presented a nonlinear trend and less sensibility respect to the second part, as confirmed in other works [6, 16, 19]. The response of SSs, conversely, evidences a good linearity, since it can be fitted by a first-order polynomial curve (R-square=0.991). In addition, the sensitivity of the SSs is quite constant over the whole range. V. PRELIMINARY APPLICATION In order to show a preliminary application of both types of sensors for the development of a sensory glove, we built a prototype, which was able to measure the movement of one finger joint. A glove (polyester/elastan mixed fabric) was equipped individually with a sample of both bend sensor types placed on the dorsal aspect of the proximal interphalangeal (PIP) joint of the index finger, without overlapping other joints. As showed in Fig. 8, the sensors were placed on the glove to cover the PIP joint similarly to the placement on the automatic setup. The flex sensor was inserted into a fabric sleeve sewn on the dorsal aspect of the glove. The sleeve allowed both the bending of the sensor and its adherence to the finger. The stretch sensor was placed on the glove by stitching its two ends directly on the fabric. A subject wore, one at time, the gloves, gradually closing and opening the hand four times. Sensor data were acquired by the aforementioned multimeter and plotted to show the relative change of the resistance value (%), with respect to the nominal value of the sensor measured in the flat position of the hand (see Fig. 8). Due to the bending of the index finger, the resistance value of the sensors increased showing a percent change up to 816% for the flex sensor and up to 68% for the stretch sensor. When the sensors were unfolded during the opening of the hand, the resistance value decreased to the initial value. As expected, the change in resistance for the flex sensor was higher than for the stretch sensor. Nevertheless, both sensors allowed the measure of the flexion/extension of a finger joint. (a) Fig. 8. A preliminary application using a prototypal sensory glove equipped with a flex sensor (a) and with a stretch sensor (b) in order to measure the flexion/extension of the proximal interphalangeal joint of the index finger. The subject gradually opened and closed the hand four times: (upper) pictures of the prototype showing the placement of the sensors and the positions of the hand during the flexion/extension task; (lower) the relative resistance change (%) of the sensors due to the movement of the hand. (b)

6 VI. CONCLUSIONS Commercially available RFSs and SSs were evaluated aiming at selecting the most suitable sensor type to track human joint flexion/extension. In particular, the experiment focused on the assessment of sensors able to track hand finger joint movements. Both types of sensors were characterized by means of a completely automated setup consisting in a motorized hinge that simulated the flexion/extension of a finger joint. The obtained results for the investigated RFSs were coherent with the findings shown in other works, confirming the feasibility of these sensors in human motion assessment applications. The selected SSs demonstrated a limited resistance variation, a higher hysteresis (in percentage), and a higher standard deviation, with respect to their RFSs counterparts. Nevertheless, SSs offer a meaningfully higher linearity. Linearity of the sensors response can play a role for the application in a sensory glove. This is because the calibration phase, necessary to pay into account users with different hand size, can become time-consuming to determine coefficients of the polynomial curve approximating sensors responses. If the sensor can be approximated by a linear curve, just two calibration points are necessary; while three calibration points are required if the approximation is made by a second-order polynomial curve. This study demonstrated that the SSs present linearity higher than the one of their RFSs counterparts, so that this result has to be conveniently taken into account. It is also important to notice that a stretch sensor can be cut so that every lengths of the sensor are allowed, just adding the electrical connections to the two ends of the sensor. This feature makes stretch sensors suitable even for small sizes of the glove and for tracking small segments of the body, as the phalanges of the little hand finger. In further studies, we aim also at evaluating dynamic performances of sensors, rather than the quasi-static characterization here proposed. In conclusion, this work focuses on the comparison between the mostly adopted RFSs and the novel SSs, which, to our best knowledge, have never been previously used for sensory gloves development. In addition, results of the preliminary application show the feasibility of both sensors in measuring the flexion/extension of a finger joint. ACKNOWLEDGMENT Authors thank Mr. M. Palmacci for helping in mechanical facilities and Eng. M. Bove for helping in measurements. Furthermore, authors would like to thank Prof. Mariano Bizzarri, Dr. Simona Zoffoli and Dr. Francesca Ferranti of the Italian Space Agency (ASI). REFERENCES [1] P. Bonato, Advances in wearable technology and applications in physical medicine and rehabilitation, J Neuroeng Rehabil, vol. 2, 2, [2] G. Saggio, A. Lazzaro, L. Sbernini, F.M. Carrano, D. Passi, A. Corona, V. Panetta, A.L. Gaspari, and N. Di Lorenzo, Objective surgical skill assessment: an initial experience by means of a sensory glove paving the way to open surgery simulation?, J Surg Educ, vol. 72, pp , [3] R.R. Coelho, S.B. Soares, L. Landau, E.H.M. Dantas, J.L.D. Alves, and G.G. Cunha, Analyzing the human body joints range of movement: the use of bendsensors, Proc. Biosignals and Biorobotics Conference (BRC), 2013 ISSNIP, 2013, pp [4] G. Saggio, and L. Sbernini, New scenarios in human trunk posture measurements for clinical applications, Proc. Medical Measurements and Applications Proceedings (MeMeA), 2011 IEEE International Workshop on, 2011, pp [5] G. Saggio, Mechanical model of flex sensors used to sense finger movements, Sensors and Actuators A: Physical, vol. 185, pp , [6] G. Saggio, F. Riillo, L. Sbernini, and L.R. Quitadamo, Resistive flex sensors: a survey, Smart Mater. Struct., vol. 25, , [7] L. Dipietro, A.M. Sabatini, and P. Dario, A survey of glove-based systems and their applications, IEEE Trans Syst Man Cybern C Appl Rev, vol. 38, pp , [8] E. Al-Fakih, N.A. Abu Osman, and F.R. Mahamd Adikan, The use of fiber bragg grating sensors in biomechanics and rehabilitation applications: the state-of-the-art and ongoing research topics, Sensors, vol. 12, pp , [9] H. Dejnabadi, B.M. Jolles, and K. Aminian, A new approach to accurate measurement of uniaxial joint angles based on a combination of accelerometers and gyroscopes, IEEE Trans Biomed Eng, vol. 52, pp , [10] L. Dipietro, A.M. Sabatini, and P. Dario, Evaluation of an instrumented glove for hand-movement acquisition, J Rehabil Res Dev, vol. 40, pp , [11] P.T. Gibbs, and H.H. Asada, Wearable conductive fiber sensors for multi-axis human joint angle measurements, J Neuroeng Rehabil, vol. 2, 7, [12] P. Strohmeier, R. Vertegaal, and A. Girouard, With a flick of the wrist: stretch sensors as lightweight input for mobile devices, Proc. of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, 2012, pp [13] F. Lorussi, E.P. Scilingo, M. Tesconi, A. Tognetti, and D. De Rossi, Strain sensing fabric for hand posture and gesture monitoring, IEEE Trans Inf Technol Biomed, vol. 9, pp , [14] G. Saggio, L.R. Quitadamo, and L. Albero, Development and evaluation of a novel low-cost sensor-based knee flexion angle measurement system, Knee, vol. 21, pp , [15] N.P. Oess, J. Wanek, and A. Curt, Design and evaluation of a lowcost instrumented glove for hand function assessment, J Neuroeng Rehabil, vol. 9, 2, [16] M. Borghetti, E. Sardini, and M. Serpelloni, Evaluation of bend sensors for limb motion monitoring, Proc IEEE International Symposium on Medical Measurements and Applications (MeMeA), 2014, pp [17] A. Tognetti, F. Lorussi, G.D. Mura, N. Carbonaro, M. Pacelli, R. Paradiso, and D. De Rossi, New generation of wearable goniometers for motion capture systems, J Neuroeng Rehabil, vol. 11, 56, [18] K.-S. Lee, and M.-C. Jung, Ergonomic evaluation of biomechanical hand function, Saf Health Work, vol. 6, pp. 9-17, [19] M. Borghetti, E. Sardini, and M. Serpelloni, Sensorized glove for measuring hand finger flexion for rehabilitation purposes, IEEE Trans Instrum Meas, vol. 62, pp , 2013.