Multi-subject/daily-life activity EMG-based control of mechanical hands
© Castellini et al; licensee BioMed Central Ltd. 2009
Received: 10 December 2008
Accepted: 17 November 2009
Published: 17 November 2009
Forearm surface electromyography (EMG) has been in use since the Sixties to feed-forward control active hand prostheses in a more and more refined way. Recent research shows that it can be used to control even a dexterous polyarticulate hand prosthesis such as Touch Bionics's i-LIMB, as well as a multifingered, multi-degree-of-freedom mechanical hand such as the DLR II. In this paper we extend previous work and investigate the robustness of such fine control possibilities, in two ways: firstly, we conduct an analysis on data obtained from 10 healthy subjects, trying to assess the general applicability of the technique; secondly, we compare the baseline controlled condition (arm relaxed and still on a table) with a "Daily-Life Activity" (DLA) condition in which subjects walk, raise their hands and arms, sit down and stand up, etc., as an experimental proxy of what a patient is supposed to do in real life. We also propose a cross-subject model analysis, i.e., training a model on a subject and testing it on another one. The use of pre-trained models could be useful in shortening the time required by the subject/patient to become proficient in using the hand.
A standard machine learning technique was able to achieve a real-time grip posture classification rate of about 97% in the baseline condition and 95% in the DLA condition; and an average correlation to the target of about 0.93 (0.90) while reconstructing the required force. Cross-subject analysis is encouraging although not definitive in its present state.
Performance figures obtained here are in the same order of magnitude of those obtained in previous work about healthy subjects in controlled conditions and/or amputees, which lets us claim that this technique can be used by reasonably any subject, and in DLA situations. Use of previously trained models is not fully assessed here, but more recent work indicates it is a promising way ahead.
Electromyography (EMG from now on) is a well-known diagnostic tool for detecting muscle disorders from motor unit activation potentials [1, 2]. In its non-invasive (surface) version it has also been used since the Sixties [3–5] to enable amputees control one or two degrees-of-freedom (DOFs) of active upper limb prostheses. Its commercial/clinical applications include, e.g., Otto Bock's SensorHand Speed , the Motion Control Hand and the Utah Arm , and more recently, Touch Bionics's i-LIMB , with 5 active and one passive DOF. In some of these cases, force/torque are also controlled.
The popularity of surface EMG stems from its cheapness, simplicity of use and non-invasiveness.
Nevertheless, research on more and more dexterous mechanical hands is ongoing (e.g., the DLR-II hand  and the Cyberhand [10, 11]) and soon a finer control will be required. To this end, at least since 2002 [12–15] it is known that a few surface EMG electrodes suffice to recognise up to nine isometric/isotonic hand postures. This potentiality has so far been exploited clinically in the i-LIMB only, and to a very limited extent so far, as far as we know. In previous work it has also been shown that a dexterous hand prosthesis can be feed-forward force-controlled while detecting grasping postures [15, 16] in real time. So it appears that plain, old EMG still has to be exploited in full.
The work presented in this paper fits in this line of research, extending previous results along two "orthogonal" directions: first, we analyse data collected from 10 healthy subjects and thus try and assess the general applicability of the technique; second, we compare a baseline controlled condition with a "Daily-Life Activity" (DLA) one, in which subjects walk, raise their hands and arms, sit down and stand up, etc., while performing the same actions of the baseline. The DLA condition is an experimental proxy of what a patient is supposed to do in real life. Lastly, we propose a cross-subject model analysis, i.e., training a model on a subject and testing it on another one. The use of pre-trained models could be useful in shortening the time required by the subject/patient to become proficient in using the prosthesis.
Materials and methods
Ten healthy subjects joined the experiment after having given their informed consent. The subjects were two women and eight men, nine right-handed and one left-handed, average age 30.9 ± 8.45 years, standard Caucasian weight and height. They were given no knowledge of what the experiment was about.
The subject freely repeated each grasping action for 100", resting for 30" in between grasps. The whole procedure was repeated twice for numerical robustness purposes. This "baseline" phase will be referred to from now on as the Still-Arm phase (SA).
Phase 2, which started soon after phase 1 for each subject, consisted in repeating phase 1 while the subject was left free to move, walk around, lift and pronate/supinate the arm and forearm, sit down and stand up from a chair. This second phase is intended as a laboratory-controlled proxy of the main movements a patient is expected to do during DLAs. This phase will be called Free-Arm phase (FA).
Each subject's experiment resulted in something more than 1200" of data. Data were sampled at 2 KHz, resulting in about 2.4 × 106 samples for each subject, equally distributed in each phase.
Equipment and electrode placement
Seven electrodes were glued on each subject's dominant forearm, according to this anatomic guideline:
on the forearm ventral side: near the wrist, above the flexor pollicis longus; centrally, above the flexor digitorum superficialis; near the elbow, above the flexor digitorum profundus; and near the wrist, above the flexor digitorum superficialis again;
on the forearm dorsal side: near the wrist, above the extensor pollicis brevis/abductor pollicis longus; centrally, above the extensor digitorum communis and extensor digiti minimi.
These positions were chosen, according to the medical  and bioengineering  literature, to detect the activity of the flexor and extensor muscles of the forearm which are most relevant during grasping. Figure 2 (rightmost panel) shows the typical electrode positioning. Notice that there may be remarkable inter-arm differences depending on the subjects' age, gender and physical fitness. Moreover, some of the aforementioned muscles are deep into the forearm, so that muscle cross-talk cannot be avoided. This is a well-known problem in the EMG literature [1, 13].
The root-mean-square (RMS) of the EMG was evaluated using a time window T RMS . The optimal value of T RMS was evaluated independently for classification of the grasping posture and force detection, via grid search, in a preliminary phase of the experiment, and set to 500 ms for classification and 100 ms for regression. The choice of the RMS, as opposed to the simpler rectification and filtering, is motivated by its well-known relationship to the force exerted by the related muscle [1, 2, 13]. Rectification plus filtering would likely work as well, and it is indeed employed in some commercial myoelectrodes such as Otto Bock's MyoBock ones .
Notice that the right choice of T RMS can be, in general, crucial: a small value will make the system more responsive (i.e., implies a smaller delay) but a higher value will be more informative and improve the performance (especially in the case of classification, as we verified). On the other hand, it is known that the EMG signal anticipates the muscle movements by a few hundreds milliseconds; therefore, in a practical application derived from this experiment, a wider lag would be more acceptable than one would expect. The electromechanical delay (EMD) of a muscle is defined as the interval between the onset of the electrical activity of the muscle (EMG) indicating its activation by the neural system and the onset of the resulting change in the mechanical variable observed. The delays reported range from 25 to 100 ms for different muscles and tasks .
Lastly, samples for which the applied force was lower than a specific threshold were removed. After verifying several choices both numerically and visually, the threshold was uniformly set at 20% of the mean force value obtained for each subject and phase.
According to previous literature (e.g., [14, 16]), the statistical analysis was carried on using Support Vector Machine (SVM). For a comprehensive tutorial on SVMs refer to [23, 24]. SVMs are a statistical learning method able to build an approximated map between an input space and a label (classification) or a real value (regression). Classification is here used to classify the type of grasp according to the EMG signal, whereas regression is used to understand how much force the subject is exerting, independently from the grasp type. The input space is ℝ7, one coordinate for each EMG electrode. We used the ground truth values as labels and the force value given by the force sensor for the regression. Notice that SVMs work here in real-time, associating a grasp type and a force value to an EMG value at each instant of time. Grasp type and forces are then predicted almost at the onset of the grasping movement, differently from what happens in other approaches (e.g., [14, 25]) in which all values of the input signal over a further time-window are employed as the input space.
In order to ease the computational burden we employed uniformisation  to reduce the size of the training sets. The samples in a training set are considered one by one in chronological order, as it would happen in an on-line setting, and each new sample is added to the training set if and only if its Euclidean distance from all training samples retained so far is larger than a predefined value d. Values of d were set to 0.02 for the SA phase and 0.032 for the FA phase. These values were chosen in order to get not more than one thousand training samples for subject 1. The choice is arbitrary, but notice that (see  again) the performance of such systems changes linearly as d changes, whereas the training set size varies polynomially; thus, it is always possible to find a polynomially smaller training set, if needed, which will degrade the performance only linearly. This really means that the initial choice of d is not crucial. Also, notice that testing sets have not been uniformised, in order to give a more realistic result.
SVM analysis was performed for each subject and for each phase, to check how the performance depends upon subjects and conditions. For classification, the performance index is, as is customary, the percentage of overall correctly guessed labels. For regression, the performance index is the correlation coefficient evaluated between the predicted force signal and the real one. The choice of the correlation coefficient is suggested by this consideration: when driving a prosthesis we are not interested in the absolute force values desired by the user/subject, since mechanical hands usually cannot apply as much force as human hands do, for obvious safety reasons (or, e.g., in teleoperation scenarios, they could be able to apply much more force than a human hand can). Rather, we are concerned about getting a signal which is strongly correlated with the user/subject's will. Anyway, we also report about the normalised root mean-square error (NRMSE), in order to give a broader view of the results. Normalisation is done against the signals' ranges (notice, though, that correlation is the criterion used to find the optimal parameters during grid search). We employed a well-known freely available SVM package, libsvm v2.83 , in the Matlab wrapped flavour; the Gaussian kernel was chosen, since it is a standard choice in previous literature. EMG data were normalised along each dimension, as is customary, by subtracting the mean value and dividing by the standard deviation. 5-fold cross-validation was used to assess the generalisation error for each training set; this measure was then used for grid-searching the typical Gaussian kernel hyperparameters of a SVM, called γ and C. Once these parameters were found, the overall performance was evaluated as the mean and standard deviation of the performances obtained on each fold.
Regression figures (middle and bottom panels) show that for the SA phase the correlation to true signal ranges from 0.9784 ± 0.0017 (subject 5) to 0.8959 ± 0.0033 (subject 8), whereas for the FA phase it ranges from 0.9657 ± 0.0022 (subject 5) to 0.8161 ± 0.0078 (subject 8). On average, the correlation is 0.93 ± 0.04 for the SA phase and 0.90 ± 0.05 for the FA phase. Again, consistency by subject and by phase appears. Remarkably, not all subjects which are slightly harder for regression (namely, 1, 2, 3, 6, 8) happen to be hard for classification; in particular, only subject 8 is definitely hard both for classification and regression, while, e.g., subject 6 is hard for regression but not that hard for classification. The bottom panel shows that an analogous situation appears if we consider the NRMSE. (Recall that the NRMSE is an error measure while the correlation to target is a positive performance index.)
Mean values and standard deviations of the hyperparameters γ and C.
-0.35 ± 0.58
1.6 ± 0.84
-0.65 ± 0.54
1.55 ± 0.83
-0.50 ± 0.24
1.45 ± 0.44
-0.60 ± 0.26
1.45 ± 0.37
FA-models tested on SA data obtain an average accuracy of 75.11% ± 12.34% for classification and 0.8056 ± 0.1151 for regression; whereas testing SA-models on FA data gives 70.17% ± 11.99% in classification and 0.7530 ± 0.1153 in regression. The advantage of FA models over SA models is apparent, uniform and consistent. Notice that here we show no error bars, since, for each subject and phase, there is just one training set and one testing set.
The Figure confirms that the training set size has a decreasing polynomial trend, while the performance changes linearly . In particular, for d = 0.032 the previously shown performance appears, whereas if a larger performance is required, one can increase the number of samples in the training set, or, which is equivalent, reduce the magnitude of d. For instance, to get an accuracy of about 90% d must be set at 0.2 ending up in a training set with some 1600 samples.
The overall results indicate that a large amount of the models overlap, or at least that there is a certain cross-subject capacity of prediction. Consider the numbers below the matrices in the Figure: in classification, the performances are 51.69% and 54.04%, with the remarkable particular that the FA-models are slightly, but consistently, better in cross-subject analysis (higher mean values and lower standard deviations) than the SA-models. As far as regression is concerned, the average cross-subject correlation is around 0.60. Notice that models trained on subjects 6 (for the SA phase) and 8 (FA phase) appear to be particularly bad in predicting other subjects' data (the related rows of the bottom left and right matrices, in turn, are rather darker than the average).
This analysis for each pair (i, j) of subjects and for the two phases and problems shows that inverse correlation is absent in the case of the FA phase in classification; it is mild (-0.32) for SA in classification; and that it is strong in the case of regression (-0.63 for the SA phase and -0.65 for the FA phase). It is likely that the correlation in regression is connected to the actual smoothness of the function the system is trying to approximate. It is unclear why the classification problems show a weak correlation or none at all.
Discussion and conclusion
can this technique be applied to any (healthy) subject?
will it work in Daily-Life Activities?
The results presented above point at a positive answer to both questions.
The first question is answered by noting that a uniformly good performance is obtained for each subject, in each phase. The figures obtained by on the SA phase are comparable to those found in other, related work such as [11, 34] or  where the predicted signals were actually used to control the DLR-II hand in real-time. This indicates that the approach will reasonably work on any healthy subject. Combining this result with the more recent results obtained on amputees listed above, one can conclude that the approach is viable for a wide range of patients, too. Notice that SVMs are by no means the only approach to solve this problem; linear regression, neural networks, LWPR  and Hidden Markov Models , among others, have been employed too, with similarly good results; probably, even simpler approaches would get an acceptable level of performance, which further raises the hopes for a real system based upon these results. From the point of view of machine learning, interpreting surface EMG is an easy task, a feeling corroborated, at least in the case of regression, by the uniformity of the optimal hyperparamters found by grid search.
The second question is here equivalent to asking whether the performance is comparable between the SA and FA phases, provided that the FA phase is a reasonable experimental proxy of DLAs of the standard patient. The results obtained in the FA phase are actually in the same order of performance as those in the SA phase. A deeper analysis reveals that FA models are in a sense "wider" than SA ones, since they test better on SA data than the reverse.
As an aside result, it turns out that uniformisation produces small training sets (about 30 times smaller than the original, subsampled sets) which are used to generate models with excellent accuracy. The phenomenon described in  is here confirmed: as the minimum distance d is linearly increased, performance degrades linearly while the training sets become polynomially smaller. This opens up the possibility of using it to build asymptotically bound training sets, which is paramount in an on-line setting, where the data flow is potentially endless.
Notice that, in this work, the training sets are, in absolute terms, small, since each subject could not be tested for more than 20 minutes; this means that the models presented here might suffer from noise introduced by medium-to-long term factors such as, e.g., muscle fatigue, sweat and/or electrode re-positioning. In  it is shown that these problems could be overcome by a sufficiently long training time, and we see no reason to believe that this is not the case here.
Also notice that, in general, predicting the grip force from the EMG signal is nothing new -- the EMG-to-force is well-known and has been modelled, among other methods, via linear regression . Our regression model is novel in that it predicts the force to a similar degree of precision independently of the grasp type employed. So it can be used in parallel with the classifier, as it has indeed been done in . As far as cross-subject analysis is confirmed, the figures presented here cannot be used in practice, although they are better than chance; but notice that in  a more refined approach has been employed successfully, indicating that pre-trained models can be effectively used to improve classification and regression performance, with respect to tabula rasa learning.
This work has been partially supported by the EU project NEURObotics, FP6-IST-001917.
- De Luca CJ: The use of surface electromyography in biomechanics. Journal of Applied Biomechanics 1997,13(2):135-163.Google Scholar
- De Luca CJ: Surface Electromyography: Detection and Recording. 2002.Google Scholar
- Bottomley AH: Myoelectric control of powered prostheses. J Bone Joint Surg 1965, B47: 411-415.Google Scholar
- Childress DA: A myoelectric three-state controller using rate sensitivity. Proceedings 8th ICMBE, Chicago, IL 1969, 4-5.Google Scholar
- Sears HH, Shaperman J: Proportional myoelectric hand control: an evaluation. Am J Phys Med Rehabil 1991, 70: 20-28. 10.1097/00002060-199102000-00005View ArticlePubMedGoogle Scholar
- Otto Bock SensorHand Hand Prosthesis2008. [http://www.ottobockus.com]
- Motion Control Hand Prosthesis2008. [http://utaharm.com]
- The i-Limb system2007. [http://www.touchbionics.com]
- Huang H, Jiang L, Zhao D, Zhao J, Cai H, Liu H, Meusel P, Willberg B, Hirzinger G: The Development on a New Biomechatronic Prosthetic Hand Based on Under-actuated Mechanism. Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems 2006, 3791-3796. full_textGoogle Scholar
- Carrozza M, Cappiello G, Micera S, Edin BB, Beccai L, Cipriani C: Design of a cybernetic hand for perception and action. Biological Cybernetics 2006,95(6):629-644. 10.1007/s00422-006-0124-2PubMed CentralView ArticlePubMedGoogle Scholar
- Cipriani C, Zaccone F, Micera S, Carrozza MC: On the Shared Control of an EMG-Controlled Prosthetic Hand: Analysis of User Prosthesis Interaction. IEEE Transactions on Robotics 2008, 24: 170-184. 10.1109/TRO.2007.910708View ArticleGoogle Scholar
- Ferguson S, Dunlop GR: Grasp Recognition from Myoelectric Signals. Proceedings of the Australasian Conference on Robotics and Automation, Auckland, New Zealand 2002.Google Scholar
- Zecca M, Micera S, Carrozza MC, Dario P: Control of Multifunctional Prosthetic Hands by Processing the Electromyographic Signal. Critical Reviews in Biomedical Engineering 2002,30(4-6):459-485. 10.1615/CritRevBiomedEng.v30.i456.80View ArticlePubMedGoogle Scholar
- Bitzer S, Smagt P: Learning EMG control of a robotic hand: Towards Active Prostheses. Proceedings of ICRA, International Conference on Robotics and Automation, Orlando, Florida, USA 2006, 2819-2823. full_textGoogle Scholar
- Castellini C, Smagt P, Sandini G, Hirzinger G: Surface EMG for Force Control of Mechanical Hands. Proceedings of ICRA-08 - International Conference on Robotics and Automation 2008, 725-730.Google Scholar
- Castellini C, Smagt P: Surface EMG in Advanced Hand Prosthetics. Biological Cybernetics 2008, 100: 35-47. 10.1007/s00422-008-0278-1View ArticlePubMedGoogle Scholar
- Aurion ZeroWire EMG electrodes2008. [http://www.aurion.it]
- Futek LMD500 Medical Load Cell (Hand)2008. [http://www.futek.com/product.aspx?stock=FSH00125&acc2=acc]
- Kendall FP, McCreary EK, Provance PG, Rodgers MM, Romani W: Muscles: Testing and Function, with Posture and Pain. 530 Walnut St. Philadelphia, PA 19106-3621: Lippincott Williams & Wilkins; 2005.Google Scholar
- Kampas P: The optimal use of myoelectrodes. Medizinisch-Orthopädische Technik 2001, 121: 21-27. [English translation from the German of "Myoelektroden - optimal eingesetzt"].Google Scholar
- Otto Bock MYOBOCK 13E200 = 50 Electrodes2008. [http://www.ottobockus.com]
- Wolf W, Staude C, Appel U: Enhanced onset detection accuracy "reduces" the electromechanical delay of distal muscles. Proc. 16th Annual International Conference of the IEEE Engineering Advances: New Opportunities for Biomedical Engineers Engineering in Medicine and Biology Society 1994, 392-393. full_textGoogle Scholar
- Burges CJC: A Tutorial on Support Vector Machines for Pattern Recognition. Knowledge Discovery and Data Mining 1998.,2(2):
- Smola AJ, Schölkopf B: A tutorial on support vector regression. Statistics and Computing 2004,14(3):199-222. 10.1023/B:STCO.0000035301.49549.88View ArticleGoogle Scholar
- Sebelius FCP, Rosén BN, Lundborg GN: Refined myoelectric control in below-elbow amputees using artificial neural networks and a data glove. J Hand Surg [Am] 2005,30(4):780-789. 10.1016/j.jhsa.2005.01.002View ArticleGoogle Scholar
- Chang CC, Lin CJ:LIBSVM: a library for Support Vector Machines. 2001. [http://www.csie.ntu.edu.tw/~cjlin/libsvm]Google Scholar
- Chan A, Englehart K: Continuous myoelectric control for powered prostheses using hidden Markov models. Biomedical Engineering, IEEE Transactions on 2005, 52: 121-124. 10.1109/TBME.2004.836492View ArticleGoogle Scholar
- Tsukamoto M, Kondo T, Ito K: A Prosthetic Hand Control Based on Nonstationary EMG at the Start of Movement. Journal of Robotics and Mechatronics 2007,19(4):381-387.Google Scholar
- Jiang N, Englehart K, Parker P: Extracting Simultaneous and Proportional Neural Control Information for Multiple Degree of Freedom Prostheses From the Surface Electromyographic Signal. IEEE Transactions on Biomedical Engineering 2009,56(4):1070-1080. 10.1109/TBME.2008.2007967View ArticlePubMedGoogle Scholar
- Mercier C, Reilly KT, Vargas CD, Aballea A, Sirigu A: Mapping phantom movement representations in the motor cortex of amputees. Brain 2006, 129: 2202-2210. 10.1093/brain/awl180View ArticlePubMedGoogle Scholar
- Reilly KT, Mercier C, Schieber MH, Sirigu A: Persistent hand motor commands in the amputees' brain. Brain 2006, 129: 2211-2223. 10.1093/brain/awl154View ArticlePubMedGoogle Scholar
- Sebelius FCP, Rosén BN, Lundborg GN: Refined Myoelectric Control in Below-Elbow Amputees Using Artificial Neural Networks and a Data Glove. Journal of Hand Surgery 2005,30A(4):780-789.View ArticleGoogle Scholar
- Castellini C, Gruppioni E, Davalli A, Sandini G: Fine detection of grasp force and posture by amputees via surface electromyography. Journal of Physiology (Paris) 2009,103(3-5):255-262. 10.1016/j.jphysparis.2009.08.008View ArticleGoogle Scholar
- Tenore F, Ramos A, Fahmy A, Acharya S, Etienne-Cummings R, Thakor NV: Decoding of individuated finger movements using surface Electromyography. IEEE transactions on bio-medical engineering 2009,56(5):1427-1434. 10.1109/TBME.2008.2005485View ArticlePubMedGoogle Scholar
- Vijayakumar S, D'Souza A, Schaal S: Incremental Online Learning in High Dimensions. Neural Computation 2005, 17: 2602-2634. 10.1162/089976605774320557View ArticlePubMedGoogle Scholar
- Hoozemans MJM, van Dieën JH: Prediction of handgrip forces using surface EMG of forearm muscles. Journal of Electromyography and Kinesiology 2005,15(4):358-366. 10.1016/j.jelekin.2004.09.001View ArticlePubMedGoogle Scholar
- Orabona F, Castellini C, Caputo B, Fiorilla E, Sandini G: Model Adaptation with Least-Squares SVM for Hand Prosthetics. Proceedings of ICRA-09 - International Conference on Robotics and Automation 2009, 2897-2903.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.