Feasibility Evaluation of a Novel Ultrasonic Method for Prosthetic Control ECE-492/3 Senior Design Project Fall 2011

Transcription

1 Feasibility Evaluation of a Novel Ultrasonic Method for Prosthetic Control ECE-492/3 Senior Design Project Fall 2011 Electrical and Computer Engineering Department Volgenau School of Engineering George Mason University Fairfax, VA Team members: Faculty Supervisor: Abdullah Al-Imran, Ira Augustus Hunt IV, and Andrew J. Nelson Dr. Siddhartha Sikdar Abstract: This research evaluates the feasibility of using a novel ultrasound method to obtain control signals from muscle contractions. This ultrasound-based system addresses many of the deficiencies of electromyography (EMG) based systems, including their inability to differentiate between deep contiguous muscles. The system has been developed based on a low-cost USB-based ultrasound probe and a novel method for identifying and differentiating muscle contraction within the body. The target location for this experiment is the muscles of the mid-anterior forearm: specifically the flexor pollicislongus (thumb flexion), and the flexordigitorumsuperficialis (finger flexion).this new method, called Ultrasound Motion Analysis via Template Matching (USMATM), detects and classifies motion by determining the change between a template (background image) and all subsequent (foreground) images. USMATM exploits the changes in muscle echogenicity which occurs as the muscles contract. The output of USMATM is a robust and reliable signal waveform, corresponding to each of the individual deep muscle s contractions. USMATM provides a strong output signal even from low quality ultrasound images and has demonstrated good tolerance to noise. In addition to the proper classification of muscular contraction, USMATM provides a good estimation of the rate of contraction by examining rate of change in pixel intensity over time, as well as a good estimation of degree of contraction by calculating the peek deviation from rest. 1. Statement of Need The post 9/11 conflict in the Middle East has the lowest casualty rate out of any major U.S. military action to date. The increase in survivability of U.S. soldiers is due to increased effectiveness of soldier s body armor and the rapid access to advanced medical treatment. This is not without consequence. There has been an observed increase in the number of soldiers who have undergone amputation. Prosthetics designed for forearm or hand replacement currently lag behind similar prosthetics designed for the leg. This is unfortunate, because the opposable thumb combined with fine motor skills of the fingers are necessary for an array of vital functions. As a result, there is a need for improvement in the control mechanism of current transradial (below the elbow) prosthesis. Existing 1

2 technology is unable to reliably utilize movements of deep and intermediate muscles non-invasively. Traditional transradial prosthetic controls do not correlate well to natural body movements (i.e., flexing the wrist closes the prosthetic hand). This is due to a radical change in the anatomy of the user and the inability to acquire reliable signals. In an effort to improve the quality of the signal obtained noninvasively, we propose a system utilizing ultrasound to view deep muscular movements to generate the control signals over time. 2. Approach There are three primary components of the system. The first component is the cuff and angle monitoring system. This element is responsible for acquisition of time-series ultrasonic images and providing finger angle data for testing and validation. The second component is the computer-based image processing and analysis system which consists of a series of complex computer algorithms that process the ultrasound images from start to finish. The third system is a verification system for testing the USMATM and ensuring the accuracy and reliability of the identification of the finger contracted, the degree of contraction, and the rate of contraction. In order to acquire reliable and repeatable ultrasound data, we have built a portable ultrasonic cuff shown in Figure 1. The system utilizes the Interson 7.5MHz single-element motor-controlled ultrasonic probe that connects to PC via USB. For obtaining repeatable results, the angle of the probe must be consistent. To this effect, single axis accelerometers were connected to a microcontroller, and output to an LCD. Using an additional absolute rotary encoder, the speed of finger movement and associated degree of contraction were measured for testing purposes to be compared with the USMATM system results. Figure 1: Ultrasonic cuff In order to control a prosthetic arm, the system must yield a control signal with information about the contraction of the target muscles. This involves three phases, as shown in Figure 2.Tthe system must go through these phases in order to monitor a contraction, identify the contraction, and output the signal information for prosthetic control. Since the success of this project depended on processing and analyzing ultrasound images, the scope of this report is focused on the development and validation of image processing and analysis algorithms. 3. Image Processing and Analysis In USMATM, the muscles are targeted through a training algorithm where the user is requested to contract a single finger continuously until the maximum point of change is identified through the Sum-Of-Difference (SOD) algorithm described later. A summary of the targeting process which includes an image array of large overlapping windows is then automatically placed over the relative position of the muscles. During the training process, the best signals are selected automatically from the array of large overlapping windows, without the need for user input. The best windows are selected by identifying signals with the largest and most well defined waveforms, the least amount of baseline drift, and consistent peak heights. Once the best window location is selected, the user is able 2

3 to choose the size of the windows used in the system (small, medium, or large). The small windows provide more specificity in the system, but are more affected by noise. If the user chooses to reduce the window size, the best window that is initially determined acts as a boundary for a new array of smaller overlapping windows. As before, the best signal is determined, but this time more than one signal is considered if it passes the signal selection criteria in the signal selection algorithm. Even with the smaller windows used (as compared with the automated target method), and training of each finger independently, cross talk remains too prevalent in the system to accurately distinguish between the Index, Middle, and Ring finger. Nevertheless, the thumb can be distinguished from the other fingers, and the middle finger can be identified during full contraction only, due to the sheer magnitude of the signal. In order to make a definitive identification of each finger across a large range of motion, more specificity is needed. Therefore, the user is displayed a video of the difference between the background and the foreground. The user is then asked to place a box around the area where he/she thinks the muscle is. The muscle should be presented as the area where the greatest amount of change occurs during muscle contractions. This finger identification window is used to provide a Boolean ID value corresponding to a finger. Figure 2: System functional decomposition, ultrasound image and corresponding forearm cross-section Figure 3: This image is the result of the SOD between the template and a frame where the user was in full contraction. The patterns of the large dark spots in this frame are the areas unique to each muscle being contracted, although some of the dark spots are common between muscles Once the user has repeated this process for all fingers, they are considered ready to use USMATM. During finger flexion and extension, USMATM takes the sum of difference of the template windows and the corresponding windows of all subsequent frames yielding a differential signal that is then related to an individual muscle. The muscles are identified by taking the SOD finger identification window and all waveforms. If a true is given by the finger identification window, the waveform is identified as belonging to the corresponding finger. The result is a classifier which differentiates between individual finger flexion/extension and minimizes cross talk in complex muscles, such as the flexor digitorumsuperficialis. Once the motion has been tied to a specific finger, information about the signal is extracted. The height of the waveform gives an estimate of the degree (amount) of muscle contraction. The width at ½ height of contraction gives an estimate of the duration of muscle contraction. The slope, taken between 20% and 80% of wave height, 3

4 provides estimation for rate of contraction. All of these factors provide an estimation of the characteristics of a given muscular contraction that is good enough for a graded control signal. Despite high-noise, low-resolution ultrasound images (compared to clinical imagery), the derived signals provide a signal to noise ratio (SNR) of at least 14 db without any additional processing. In fact, USMATM is able to provide such good signals that the only filter applied to suppress noise is a 5-tap moving average filter to suppress high frequency noise. This is applied so that the peaks and troughs of the signal may be determined more easily. Also, when generating Boolean values for finger identification, the input waveform is squared to give preference to the contraction event for easy identification. Figure 4 (left diagram) shows the results of the large window SOD. It is evident that this method is able to distinguish between the flexor pollicislongus and the flexor digitorumsuperficialis, but it is impossible to distinguish between the index, middle, and ring fingers with this method alone. It is difficult to distinguish between these three fingers because they are all controlled by the flexor digitorumsuperficialis. The right diagram shows results of the user specified small targeted windows. It provides a Boolean value signal which enables a rapid determination of the thumb and the middle finger contractions. Once these two muscles are identified, the ring and index can be identified by comparing the signals to the middle finger and thumb signals. Figure 4: Results of the large window SOD (left diagram) and small window SOD (right diagram) Figure5: SOD algorithm based on 9 overlapping windows, and sample best SOD waveforms 4

5 Finally, the individual muscles are located using an SOD algorithm applied via 9 overlapping windows as shown in Figure 5. The SOD algorithm works by extracting regions of interest (ROI) from the template at locations determined with respect to the centroids of the radius and ulna. The ROIs of the templates are then subtracted from the corresponding ROIs for all subsequent frames of the image. The ROI with the greatest SOD change is most likely the ROI containing the target muscle. Finally, the SOD of the most likely ROIs is presented to the user so that the best signal may be selected. The result of the new startup algorithm is a system with fewer requirements on the user and greater repeatability that is less prone to user error. Now the user could be provided with a few good wave forms, all of which could work, and they could choose between them as shown in Figure 5. The next stages of the ultrasonic design process to be completed are to extend the SOD over a ROI to other muscles via the same methods as described above. Additionally, control parameters for each waveform are extracted to represent the amount, duration and rate of muscle contraction as shown in Figure 6. Figure 6: The width at ½ height and the height of the contractions will be measured at full height of the individual wave forms. The slow contractions should be wider with a smaller peak, while the fast contractions will be tall and thin. This method provided good results as long as the bones were clearly in frame. The approach utilized large regions to determine muscle contraction which provides the advantage of insensitivity to noise, and a resistance to baseline shifts caused by slight changes in probe position or arm orientation. One other major advantage to using large regions in USMATM is that the data produced contains information about the degree of contraction or the rate of contraction which can be used to create a dynamic output signal. The disadvantage to the large window size is the system is sensitive to crosstalk corresponding to other muscles contractions. This crosstalk can cause false positives or false negatives during signal classification. Unfortunately, this method of auto targeting is not able to be specific enough to use windows small enough to reduce cross talk to a level where muscle identification can occur regularly. 4. Experimentation The testing phase was divided into three parts: (i) system repeatability, (ii) comparison of the signal obtained via the ultrasonic system against the finger verification system s results, and (iii) relation of the ultrasonic control signal to rate and duration of contraction. Test cases included: (1) All fingers moved in random succession to identify individual fingers, (2) Fingers moved individually to measure the rate of contraction, and (3) Fingers moved individually to measure the degree of contraction USMATM is able to determine the rate of contraction quite accurately (see Figure 7 and 8) by relating the slope of the rising edge (taken between 20% and 80% of max height) of the signal to the rate of contraction. This method gives a good estimation for the rate of contraction although relatively high variance is noted between slopes of contractions of identical speeds. The various relationships between slope and rate of contraction are shown in Figure 7. 5

6 Figure 6: Testing rate of contraction (notice a correlation between the rate of contraction and the slope and the variance between contractions of identical speeds in slopes 4, 5, and 6 of the slow to fast incremental plot) Figure 7: Correlation between the actual velocity and the rate of pixel variation in the USMATM system USMATM is able to determine the degree of contraction quite accurately (Figure 8 and 9) by relating the peak of each wave form with the degree of contraction preformed. This method provides a good estimation for the degree of contraction, although some large variance is noted between peeks of contractions of identical degree. This relatively large variance is not significant when compared with the total range of the values that relate to varying degrees of inflection. Figure 8: Correlation between the degree of contraction and the maximum inflection and that the variance between contractions of identical degrees of inflection can be observed. 6

7 Figure 9: Correlation between the actual velocity and the rate of pixel variation in the USMATM system. We believe that much of the variance in the degree of contraction is the result of the individual, flexing their muscles to differing degrees. We also believe that the relatively large variance in the degree of contraction is the result of the excessive noise in the system resulting from the probes shimmy. The USMATM system can reliably distinguish between the motion of separate muscles (Figure 10) such as the flexor pollicislongus and the flexor digitorumsuperficialis through a simple target identification and SOD algorithm. In order for USMATM to distinguish between the motion of complex muscles such as the flexor digitorumsuperficialis which controls the index, middle, and ring fingers, more specific targeting is needed. USMATM instructs the user to specify a small targeted window. This window provides a Boolean value signal which enables a rapid determination of the thumb and the middle finger contractions. Once these two muscles are identified, the ring finger and the index finger can be identified by comparing the signals to the middle finger and thumb signals. The resulting output of the large windows SOD after selectively choosing the waveforms corresponding to a muscular contraction provides accurate identification of the signal and maintains the information about the rate and degree of contraction. Figure 10: A compilation of the four separate signals corresponding to the Thumb, Index, Middle, and Ring fingers. 6. Summary The thumb, index, and middle fingers were able to be correctly identified by a noninvasive means of viewing deep and complex muscles of the forearm. In addition to just identifying the finger, an estimation of rate of contraction, as well as degree of contraction was also provided. All of these features were achieved using a portable, relatively low cost, low resolution ultrasound probe. Future work aims at the development of a control system which provides accurate and reliable results. While the project was mainly a demonstration of concept, it could potentially lead to a new breed of prosthetics to be developed. 7