- Research
- Open access
- Published:
Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses
Journal of NeuroEngineering and Rehabilitation volumeĀ 21, ArticleĀ number:Ā 57 (2024)
Abstract
Introduction
Despite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of usersā own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on usersā experience regarding embodiment and their interaction with different functional levels.
Methods
To investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey.
Results
Among the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented.
Conclusions
Our work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.
Introduction
Missing an upper limb not only hinders daily activities but also affects individualsā autonomy and their active engagement in societal participation [1]. Artificial limbs aim to replace a body extremity through technological solutions, effectively mitigating significant impairments in sensory-motor capabilities and facilitating physical interaction with the environment. Recent technological progress has enabled the creation of increasingly effective artificial upper-limbs [2], which can improve the quality of life of individuals with limb loss. Despite the advanced dexterity and grasping capabilities that can be achieved by modern hand prostheses, they are often not perceived by the user as part of their own body, as described in [3, 4] as Sense of Embodiment (SoE). As the clinical translation of neuroprostheses capable of restoring lost extremities progresses towards accessibility [5], the recognition of prosthetic embodiment as a crucial factor for evaluating psycho-social outcomes becomes increasingly evident [6, 7]. Theoretically, robotic limbs can achieve high precision and strength. Nevertheless, their actual performance is substantially limited by challenges associated with bionic interfacing and bidirectional transfer of motor and sensory information between the prosthesis and the user [8].
Surface electromyography (sEMG) is the traditional interface for motor decoding and bionic interfacing, and so far the gold standard in the academic environment as well as in the clinics. The clinical standard relies on two sEMG channels and a direct control approach, with certain limitations in systems with multiple degrees of freedom (DoFs). A promising alternative is based on pattern recognition methods, that characterize the signals of muscle groups with selected features. These methods often employ 6ā12 bipolar sEMG electrodes, enabling effective classification of various grasp types, which can allow more intuitive control [9, 10]. While classification techniques guarantee high robustness within each identified class, this approach hardly allows simultaneous control of multiple DoFs. To address this goal, researchers have been exploring proportional control approaches, which enable the simultaneous and proportional decoding and control of multiple DoFs through regression methods [11]. Different than classification, regression is a machine learning method that allows continuous instead of discrete predictions. Furthermore, recent development of high-density sEMG technology [12] has brought remarkable advancements regarding accuracy and decoding robustness, particularly in addressing electrode shifts. Moreover, previous research has shown that myoelectric control, i.e., provided with sEMG interfaces, and understanding of technological functions can be improved by training [13]. Among alternatives, virtual training methods have proven to be a powerful tool due to the exploration of different multi-sensory cues and immersive environments [14, 15].
Significant research has been conducted on upper-limb prostheses as essential aids for restoring both aesthetic appearance and functional autonomy. However, notable gaps persist in understanding the precise criteria required to achieve a satisfactory embodied experience [16, 17]. Embodiment has gained significant importance in prosthetics research, serving as both an indicator of technological advancements in prosthetic devices and a measure of user acceptance [6]. For instance, significant improvement in prosthesis embodiment and reduction of phantom limb pain was observed in a long-term amputee through sensory feedback delivery [18]. Although multiple definitions exist in literature, three different components for the sense of embodiment, namely Ownership [19], Location [20], and Agency [21], were proposed in [22] for virtual reality studies. However, capturing and quantifying this multi-dimensional concept still poses a complex challenge, especially in studies involving users in the loop (e.g [13, 23]). In the field of prostheses research, embodiment is typically approached from two perspectives: body representations and experimental phenomenology [6]. Many studies evaluate embodiment mostly relying on surveys, or in combination with other psychological measures, such as the proprioceptive drift (PPD) [24, 25]. In the context of the rubber hand illusion [26], the PPD measures the displacement between the perceived location of the usersā arm (not visible from their perspective) and the actual position. While PPD is usually conducted in physical environments, it has been also adapted into virtual setups [27]. Furthermore, other works have been exploring the sense of embodiment in immersive virtual reality through questionnaires [28,29,30] but without additional objective assessments (e.g. physicophysical or experimental phenomenological metrics). Note that immersive virtual reality creates a three-dimensional environment via a Head-Mounted Display and features precise tracking motion systems. Moreover, haptic interfaces may also be integrated in some occasions to offer users tactile feedback. The progress in modern control techniques opens up possibilities for improved and more adaptable bionic interfaces [31], e.g. based on machine learning algorithms, which may enhance the sense of embodiment.
This work aims to investigate the relationship between sEMG interfaces and both functionality and embodiment, including its subcomponents according with [22]. In particular, we focus on the number of sEMG sensors, time delay, and myoelectric control training. To this end, we implement a 2 degree of freedom (DoF) regressor that enables simultaneous control of hand closing/opening, and the pronation/supination of the forearm. The experimental protocol involves 24 able-bodied participants and one prosthesis user that control an upper-limb prosthesis in an immersive virtual reality environment. The controller is tested using four different conditions, including two different numbers of sEMG sensors, and a time delay to the display, similar to [32]. The sense of embodiment is quantified by an adapted version of the proprioceptive drift (PPD) test into an immersive virtual reality setup that quantifies the perception of self-location. The study includes the evaluation of subjective embodiment using a standardized survey, inspired by [32]. Additionally, functionality is assessed through the Target Achievement Control (TAC) [33]. We hypothesize that:
- H1:
-
Control using a larger number of sEMG sensors yields improved functionality compared to fewer sensors.
- H2:
-
Control using a larger number of sEMG sensors enhances subjective embodiment compared to fewer sensors.
- H3:
-
Time delay reduces the embodiment of myoelectrically controlled devices.
- H4:
-
A positive relationship exists between functionality and subjective embodiment.
Materials and methods
This study evaluates the effects of different sEMG interfaces on the sense of embodiment perceived by the user and on the functionality achieved by the user. The study is designed in two stages (shown in Fig.Ā 1), the first stage includes 2Ā \(\times\)Ā 2 repeated-measures for the sense of embodiment. In the second stage, which is designed as a between group study, we conduct pre- and post-training assessment for the functionality. The evaluation is conducted in an immersive virtual reality (VR) environment, where the user has a first-person visual perspective and can control a virtual prosthesis to interact with several objects. For this purpose, custom gaming and training scenes were designed to motivate and help the user familiarize with the VR environment and control strategy. The training environments are based on activities of daily living, such as moving objects with different shapes and dimensions from one table to another or opening a door. Selected assessments to measure the functionality [33] and the sense of embodiment [24, 34] have been adapted from literature and implemented in the VR environment. More details on the experimental setup, protocol, VR environments, and control method are presented in the following sections.
Conditions
The factors included for the Sense of Embodiment (SoE) analysis are the number of sensors (Sensors) and the time delay (Delay). The interaction between them (Sensors \(\times\) Delay) are our four conditions, which have been defined to observe the effect of different sEMG control interfaces on functionality and sense of embodiment. For Sensors, participants can experience high density (HD) or low density (LD) while for Delay, the virtual hand movement can be synchronous or asynchronous. Note that the terms HD and LD are contextualized within our protocolās terminology for high and low density, respectively, and are not directly aligned with other literature uses. Specifically, the selected conditions are:
- C1:
-
Asynchronous, LD
- C2:
-
Synchronous, LD
- C3:
-
Asynchronous, HD
- C4:
-
Synchronous, HD
Where LD (low density) indicates the case where only 4 sEMG electrodes are used, and HD (high density) employs 16 electrodes. The number of 16 electrodes were chosen based on the findings of [35], which show that a number of 16 electrodes did not deteriorate the performance compared to 192 electrodesācommon sensor configuration for high-density terminology, used in literature. Thus, we assumed that 16 electrodes can serve as a representative setup for high density sEMG interface. Synchronous is defined as the condition with the shortest possible delay between the input command-reading, and the execution visualized on the virtual hand. The asynchronous condition introduces a time delay of 500 ms adopted from [32] and is supposed to be a recognizable delay [36].
Participants
Twenty-four able-bodied participants took part in the experimental evaluation (8 female, 16 male; 3 left-hand, 21 right-hand dominant, average age of 24.1 Ā± 4.1 years). All participants used their dominant hand during the experiments to maximize the control performance and ensure optimal muscle condition. Nine participants had previous experience in EMG control while twelve used VR commercial games. A prosthesis user is also included in the experimental evaluation for reference and discussion regarding the applicability of the findings to the target population. This individual is a 35-year-old male with a transradial amputation of the right arm. He has no prior experience with VR or the specific myoelectric control method used in this study. All participants were naive to the specific VR environments developed and used for this study, the particular control method adopted, and the experimental protocol. All participants were in a good physical and mental condition and gave their informed consent. This study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Commission of the Technical University of Munich, reference number 478/21Ā S-SR (27.09.2021).
Experimental setup
EMG setup and data collection
The participants were equipped with two Myo sEMG sensor armbands resulting in a total of up to 16 sensors (see Fig.Ā 2). The armbands were placed on the muscle bulge of the participantās forearm, approximately 1Ā cm below the elbow. The upper armband was placed with channel number 4 (see Fig.Ā 2) on the extensor digitorum muscle. The sEMG sensors were located similarly for the prosthesis user, covering his entire residual limb. The sEMG data was used to control the closing and opening of the hand, as well as pro/supination of the forearm. For that purpose, the data of the sEMG armbands was acquired at 200Hz via Bluetooth Low Energy (BLE) in a C# software called Interactive Myocontrol developed by DLR for biosignal data acquisition and training. During a training phase, sEMG data were recorded while the participants were instructed to reproduce the movements of a 3D hand model, as shown on a screen in front of them (outside of immersive VR). The trained gestures were: rest, power grasp, pronation, as well as the combination of pronation and power grasp. The default position of the hand, namely the rest position, was with the palm facing upwards and hand completely opened with relaxed fingers. All trained gestures started with the default position and move to the targeted configuration. Therefore, the trained gesture of pronation covered the full range of forearm rotation, avoiding the inclusion of supination as a trained gesture. Five repetitions of this sequence of gestures were recorded: the first three in a static position, with an elbow angle of 90\(^{\circ }\), and the last two dynamically, i.e., while moving the arm around to simulate VR conditions. After a second order Butterworth low-pass filter with a cut-off frequency of 1Hz, the data was then sent to a machine learning algorithm, namely Ridge Regression with Random Fourier Features (RR-RFF), which has already been used successfully online for myocontrol in several experiments [37, 38]. Once the machine learning model was trained, the prediction of the regressor was sent via User Datagram Protocol (UDP) to the VR environment in order to move a VR hand/wrist model.
VR setup
The virtual environment was created using Unity3D (version 2020.3.22, Unity Technologies Inc., San Francisco, USA) on a Windows 10 operating system. The participants were using an HTC Vive Pro head-mounted-display (HMD) and a HTC Vive tracker as visualization input, and VR control. The tracker, used to detect the lower arm position, was placed on top of the more distal Myo armband, similar to [39]. The HMD was used to control the position of the shoulder, which is located 15Ā cm below the eye and 20Ā cm to the left or right (accordingly to the anthroprometric tables) of the head center. The user had a first-person visualization of one fully controllable arm, which simulates the virtual prosthesis and can be used to interact with the VR environment. The final setup of one participant wearing the HMD, the two Myo armbands (Thalmic Labs Burlington, VT, USA) and the tracker is shown in Fig.Ā 3.
Experimental protocol
Before starting the experiment, the participants are equipped with two sEMG Myo armbands and an HTC Vive tracker (HTC Corporation, Taoyuan, Taiwan) and are asked to stand in front of the computer. The experiment then begins with the sEMG data collection and model training. After, a brief visual test was conducted by the experimenter to ensure the participant is capable of performing the different movements. The participant wears the VR HMD and visualize the first VR environment. The participant is immediately guided in the first embodiment assessment environment to collect the data that will be used as baseline. The baseline represents the sense of embodiment that the user has with the least amount of time seeing and interacting with his virtual arm in VR and without any controller experience. The experiment continues with the Interaction phase, conducted in a gaming environment for 5Ā min, after which the sense of embodiment is evaluated. This first part of the experiment is repeated four times, one for each condition in a randomized order (the condition sequence is determined by randomized latin squares). The participant has no information about the tested condition. Once all four conditions have been evaluated, and after taking a 10-minute break, the participant continues with the second stage that focuses on the functional assessment. To limit the duration of the experiment to under 2Ā h and avoid fatigue, each participant experiences only one condition for the functional evaluation. Only synchronous conditions are tested since we consider them the most relevant for the functional assessment. The participants (N = 24) are divided into two equally distributed groups (N = 12), which tested condition C2 or C4. Functionality is assessed before and after 10-minutes training. Five training environments were developed for the participants to experience and practice the control modality. A schematic of the whole experimental protocol is shown in Fig.Ā 1. A prosthesis user conducted the same experimental protocol. However, time constraints and fatigue prevented him from conducting the second part of the experiment (Stage 2āFunctionality). For this reason, prosthesis user results do only concern embodiment assessment.
Virtual reality environments
The VR environment was designed to include a total of nine different scenes, three for the assessment (functionality and sense of embodiment) and six for training and familiarization. An initial room allowed the user to select among all the different scenes by holding his arm into the specific scene widget. The participants were able to move freely in each scene and were guided by the experimenter.
Gaming environment
The gaming environment was designed to allow the participants to familiarize themselves with the virtual reality setup and the control method in an engaging activity. In the game, the participants were placed in a virtual natural landscape and were asked to control the virtual arm and hand to pick up one of the three water guns placed on a wooden table in front of them. Each water gun had a different color (red, yellow, and blue) and would be used to break the constantly falling water balloons with a corresponding color. If the participant hit a balloon with a color that matched the one of the water pistols, the balloon disappeared, and the user collected one point. The score and a countdown timer was shown in the top-center of the usersā field of view. Thereby the user was motivated to switch the water guns continuously in a random order to match the colors of the appearing balloons. The movements used to pick up the water pistols were forearm pro-/supination and hand opening/closing. Triggering was active as long as the water pistol was held. The user was free to move around in the scene, and balloons were spawned at Ā± 90\(^{\circ }\) radius from the users starting position. FigureĀ 4 shows the implemented scene.
Sense of embodiment assessments
(1) Virtual Arm Proprioception Error (VPE): While there is no consensus in the community on a standardized clinical assesments to measure quantitatively the sense of embodiment [6], the proprioceptive drift (PPD, Fig.Ā 5a) test is commonly used in literature as a quantitative measure for the sense of embodiment, e.g., especially in studies focused on the rubber hand illusion [41,42,43]. While this test is mostly conducted in a physical environment, previous works already implement the same principle in non-immersive VR settings by enforcing a position offset with respect to the participants real arm [27, 44].
The accurate proprioceptive awareness of our body location is fundamental for effective interaction with the environment, and a distorted representation of our arm which might hamper dexterous manipulation skills. This aspect deserves careful consideration, especially in the field of prosthetics and motor disabilities. In this work, we implemented a modified version of the PPD test for our immersive virtual environment that evaluates the self-location perception. In the original PPD test, the rubber hand (\(P_{vh}\) in Fig.Ā 5a) as well as the real hand (\(P_{rh}\) in Fig. 5a) are occluded and the participants are asked to identify the perceived location of their real hand using the contralateral side or a voice command. In the PPD, the common distance between the rubber, or virtual hand lies in the range of 15ā30Ā cm [44,45,46]. Unlike in the PPD, we measured the distance between the perceived position of the virtual arm (when not visible in the screen) and the actual location of the virtual arm, instead of the real arm. This was decided because our main interest lies in the proprioception evaluation of the virtual hand related to the tested condition. Hence, we did not deliberately introduce an offset distance to the position of the virtual hands in the modified version of the PPD used in our study. Nonetheless, there was still a slight discrepancy between the position of the virtual hand and the real hand of the subjects. This variability falls within a range of 5ā10Ā cm, influenced by factors such as the individualās lower arm length, hand dimensions, and the tolerances of the tracking system. The scene was implemented in a plain black environment to avoid potential distraction and maintain the focus on the task. The participants entered the room and saw their virtual arm for 3Ā s before it vanishes. From then, the only visible element was a red sphere that moved from left to right on a horizontal plane at a speed of 0.5 m/s, placed at the height of the users hand. The participants were asked to say āstopā when they perceived that the ball had reached the center position of the virtual hand (see Fig.Ā 5b). If participants were not able to provide this information on a first trial, the test restarts immediately. The distance between the perceived virtual arm (corresponding to the location of the red ball when the participant says stop), and the actual position of the virtual hand (noted as \(P_{vh}\)) was then defined as the virtual arm proprioception error (VPE). We expected that the intuitive knowledge of the virtual hands position and thereby the distance between virtual hand and perceived virtual hand changes depending on the condition. The VPE was measured in Unity units, which have the scale 1 Unity unit = 1Ā m (100Ā cm). As described in section 2.4, a baseline measurement was performed immediately after entering the VR which was then included in the VPE measurements for the conditions (section 2.1) as described by Eq.Ā 1 (\(C \in\) [baseline, LD asynchronous, LD synchronous, HD asynchronous, HD synchronous]) and Eq.Ā 2.
(2) Questionnaire: As an additional evaluation of the subjective embodiment, we administrated an established questionnaire, adapted from Longo et al. [34], due to its widespread familiarity and frequent use in the literature. This questionnaire is a widely used, psychometric tool based on ten statements rated on a 7-point Likert scale ranging from 1 (= strongly agree) to 7 (= strongly disagree). Similarly to the approach by Frƶhner et al. [32], we adapted the items by selecting all items except item 8, which is not applicable to our study since we have no sensation of touch in the VR. The selected items are shown in Table 1. The questionnaire was implemented as one scene in the VR environment to avoid interruptions and potential reduction of the immersiveness while removing the VR HMD. This environment was designed as a plain black room with the questions displayed in the central area together with the 7-point Likert scale.
Training environments
To study the effect of the training within the proposed conditions, additional VR scenes were implemented with the specific goal of encouraging the participant to explore the control method and practice. For each scene, we included tasks that require simultaneous control of hand closing/opening and forearm pro-/supination movements. The environments were inspired by activities of daily life (ADL) and standard assessment for upper-limb impairments. The five scenes implemented are presented in Fig.Ā 6. In two of the scenes, the participant was asked to open a door. In the first case (Fig.Ā 6a), the door had a standard door handle that should be pressed down and rotated to open the door. In the second scene (Fig.Ā 6b), the door was locked and the key was placed on a table next to it. The participant was asked to pick up the key, insert it into the lock and rotate it by 90\(^{\circ }\) to open the door. In the third scene (Fig.Ā 6c), the participants had to grasp a cup filled with balls by the handle and pour them into a bowl placed on a table. This task focused primarily on training the full range of forearm pronation movement needed to completely empty the cup. To simulate activities of daily living in a pick and place scenario, the fourth scene reproduced the Clean Sweap task from the Cybathlon 2020 Global Edition [40] (Fig.Ā 6d). Similarly to the physical task, this scene presented two tables and eight different objects that should be grasped from one table and placed in a specific location on a second one. Finally, the last training scene implemented a virtual version of the Box and Blocks (Holser and Fuchs 1957), where the participant was asked to move as many red blocks as possible from one box to another (Fig.Ā 6e). In this case, we did not consider the number of blocks moved as scoring, but we let the participant play freely with the task.
Functionality test environment
A 3D version of the target achievement control (TAC) test [33] was implemented as a standard method to assess real-time myoelectric control functionalities. During the TAC test, participants had to move their virtual hand and forearm to randomly defined target positions. The task parameters were modified to 30 target positions for combined hand closure and wrist pro-/supination positions. The time to reach the target position was set to 15Ā s. If the participant was not capable to reach the position within the time frame, the target counted as failed and the next target appeared. The time the virtual hand had to be held in the target position was set to 1Ā s. The tolerance between the target position and the actual position was selected to 15%. The measured metrics were the success rate (SR), the time needed for the participant to reach the target positionātime to target (TtT), and the accumulative time the participant was in the target position while not holding it there for 1Ā sātime in target (TiT).
Statistical analysis
Data normality was evaluated by ShapiroāWilk test. For non-parametric data, such as for the likert-scale scores in the subjective embodiment, a Quantile Transformation was applied to convert the numerical data to following a Normal distribution. The statistical analysis was performed using a repeated measure analysis of variance (ANOVA, type III), with experiment-depending parameters as factors; named as Sensors Delay and Training. Differences were considered statistically significant for p-values below the threshold of \(\alpha\) = 0.05. A post-hoc Tukey test was performed after ANOVA to extract significance among factor interaction conditions. Means are reported as M: mean Ā± standard error.
Results
In this section, we present the results of the embodiment metrics (VPE and subjective embodiment score), as well as the functionality measurements from the TAC metrics. The Shapiro-Wilk tests reported not normally distributed data for the questionnaire score, the VPE, the TAC test TtT, and TiT metrics (p < 0.05), and normal distributed data for the TAC SR metric (p = 0.128). After the data transformation for not normally distributed measurements, a two-way ANOVA was used to determine relevant differences for all three dependent variables, i.e., subjective embodiment score, virtual arm proprioception error and task performance.
Embodiment results
All statistical results for the evaluation of the Sense of Embodiment are presented in Table 2. FigureĀ 7a presents the \(VPE_C\) values without taking the baseline into account and Fig.Ā 7b the VPE results according to Eq.Ā 2 (the difference from each condition to the baseline). Even though without significance, a trend in smaller VPE values for the HD conditions (sync: 6.1 cm, async: 6.3 cm) compared to the LD conditions (sync: 8.2 cm, async: 9.8 cm) is observed (Fig.Ā 7). Moreover, among LD conditions a trend of smaller VPE values is present for the synchronous condition (8.2 cm) compared to the asynchronous condition (9.8 cm). Additionally, results from the prosthesis user are displayed with red diamonds in Fig.Ā 7. The observed differences among experimental conditions for able-bodied subjects persist for the amputee, with a more distinct contrast between Delay conditions, visible also in the controllers with HD. Note that \(VPE_c\) is equitable in both Baseline and HD Sync.
Regarding subjective experience, Table 2 reports Embodiment, which represents the questionnaire ratings for all 9 Items for each participant and condition. To examine where exactly the differences between conditions are, we analyze scores provided to the subscales Ownership, Location, and Agency. Four ANOVA Test with the factors Sensors and Delay were calculated to explore whether the conditions differ from each other. Contrary to the results of the VPE, the number of sensors resulted significant (p \(=\) 0.012) for the subjective embodiment score. Furthermore, while no significance is obtained regarding the factor Delay, the interaction between Sensors and Delay (p \(=\) 0.048) shows a significant effect, with results reported in Fig.Ā 8c. Significantly higher subjective embodiment, i.e., lower score, is shown in condition C4 (HD sync; M: 3.509 Ā± 0.105), compared to conditions C1 (LD async; M: 3.856 Ā± 0.102) and C2 (LD sync; M: 3.935 Ā± 0.1), as well as in condition C3 (HD async; M: 3.537 Ā± 0.096) with respect to C2 (LD sync). Overall, the VPE presents a lower estimated mean for the HD condition (6.1 cm, 6.3 cm) than for the LD condition (8.2 cm, 9.8 cm), which is in alignment with the stronger subjective embodiment for HD (M = 3.523 Ā± 0.071) than for LD (M = 3.896 Ā± 0.071), as can be seen in Figs.Ā 7 and 8c, respectively.
FigureĀ 8a presents the answers to the subjective embodiment survey separated among the three subscales: Ownership, Location and Agency, and according to each condition tested C1āC4. Table 2 proves that the number of sensors are significantly relevant for the ownership components. Results of the corresponding post-hoc Tukey test are presented in Fig.Ā 8b to compare scores from the three subscales. These show significantly higher score (p < 0.0001) between perceived ownership (M: 4.08 Ā± 0.069) and location (M: 3.255 Ā± 0.09), as well as between perceived ownership and agency (M: 3.24 Ā± 0.102). FigureĀ 8a reports also results from the prosthesis user with red diamonds. Differences among experimental conditions are more pronounced compared to results from able-bodied subjects, with LD Async obtaining the worst scores and HD Sync the best. Although HD exhibits consistent scores among subcomponents of embodiment, variations are observed in LD, underscoring how sEMG interfaces have a strong relationship with Agency.
Functional results
For the third dependent variable, regarding task performance, three two-way ANOVA were calculated. Table 3 reports their results according to the three TAC metrics accounting for the assessment of functionality: success rate (SR), time to target (TtT) and time in target (TiT). Better task performance is associated to higher SR, lower TtT and higher TiT. In this case, factors included in the analysis are: number of sensors (Sensors), pre- and post-training outcome (Training), and their interaction (Sensors \(\times\) Training). Note that for the functionality assessment the number of participants experiencing each control condition is N = 12.
Significance is present on the Success Rate (SR) and in relation to the number of sensors (p \(=\) 0.012, see Fig.Ā 9a), with LD reporting a \(M_{SR}\) of 28.33% Ā± 3.639 and HD a \(M_{SR}\) of 51.53% Ā± 5.324. Further statistical significance (p < 0.01, see Fig.Ā 9c) is observed between C2 (LD) pre-training (\(M_{SR}\): 23.61% Ā± 4.233) and C4 (HD) post-training (\(M_{SR}\): 55.6% Ā± 8.096). A non-significant trend in higher SR scores is visible from Fig.Ā 9b for the training factor.
Regarding the time to reach the target (TtT), the ANOVA results showed no significance for both Sensors (p \(=\) 0.061) and Training (p \(=\) 0.584) factors. Nonetheless, HD and post-training conditions showed better timing performance, with \(M_{TtT}\): 8.371 s Ā± 0.413 for LD, \(M_{TtT}\): 7.046 s Ā± 0.251 for HD and \(M_{TtT}\): 8.26 s Ā± 0.34 for pre-training, \(M_{TtT}\): 7.157 s Ā± 0.36 for post-training (see Fig.Ā 9d and e). The interaction between factors (Fig.Ā 9f) showed higher TtT for C2 (LD) pre-training (\(M_{TtT}\): 8.899 s Ā± 0.483) compared to C4 (HD) post-training (\(M_{TtT}\): 6.471 s Ā± 0.169) (p < 0.01).
Results from the time in target (TiT) are presented in Fig.Ā 9gāi. Even though no significant differences are obtained among factors or conditions, we observed a similar performance trend to TtT, with longer times (better performance) in the consecutive order from C2 (LD) pre-training to C4 (HD) post-training, as well as from C2 (LD) to C4 (HD) and pre- to post-training individually.
Finally, we investigate the relation between the changes in functionality through high and low density interfaces and subjective embodiment. FigureĀ 10 shows the result of plotting the subjective embodiment against the TAC SR before training (representative for the functionality closest to during conditions), for LD (blue) and HD (orange). Participants using the HD interface achieved higher values of both aspects, with a higher slope (0.043) compared to the LD interface (0.018). This indicates a positive relationship between the higher subjective embodiment percevied through a higher functionality.
Discussion
Only non-significant variations in the virtual arm proprioception error (VPE) were observed across the different tested conditions (Fig.Ā 7 and Table 2). This might be due to the small distance between the virtual hand and the real hand position which was intentionally not set to distinct values as occures in the classic PPD, and thereby limits the phyiscal possible drift of the virtual hand dependent to the condition. Furthermore, the absence of contextual cues in immersive VR environments may negatively impact participants proprioception, as raised in [47], and decrease the measurable effect in the VPE.
Regarding the subjective embodiment, the number of sensors showed a significant effect. Participants experienced a stronger subjective embodiment for high density with respect to low density control methods, independently of the delay implemented. Contrary to what we hypothesized in H3, and what other studies have found [32, 48], the time delays did not present a strong effect. For instance, LD synchronous reports the highest score (lowest embodiment) in Fig.Ā 8c, suggesting a potential inability of perceiving a time delay when the control quality is poorer. Furthermore, the differences between the synchronous and asynchronous control conditions might be explained by the used control modality. In our study, the position of the prostheses is simultaneously and proportionally controlled with sEMG interphases, usually less precise than other control modalities, such as infrared cameras [27] and tracking or mechanical manipulators [32]. Even though sEMG are intuitive solutions for humanāmachine interfacing, when controlling a system through biosignals, there is no specific feedback from the control source to the user. Thereby the user may not be aware if the delay occurs because of the control algorithm or because of the condition. The results from the TAC metric TtT (in Fig.Ā 9dāf supports this explanation, as we can observe that the average time the participants require to reach a desired position is 7Ā sā8.2s (LD, HD, respectively, and consistent with [35] for 2 DoFs). Even though the implemented delay of 500ms should be recognizable [36], the TtT results are considerably higher than this.
Results depicted in Fig.Ā 8b demonstrated higher agreement, i.e. lower score, with the location and agency components of the survey compared to the ownership component. Our findings of a stronger perceived Agency component (see Fig.Ā 8b), and the Ownership being the weakest perceived component are in alignment with the observations from [30]. Considering the Agency component (Fig.Ā 8a), we observe that only for HD conditions, the synchronous condition achieve a lower score, corresponding to a stronger subjective embodiment. Overall, it is intuitive to perceive a change in Agency from a time delay since it directly affects the responsiveness of the control.
These results highlight the persistent challenges associated with perceiving an external arm as oneās own in VR environments, independently of the sEMG configuration used. Finally, for this preliminary study, the physiological dimensions of the virtual arm, as well as its positioning in reference to the HMD, are not adapted to the individual participant which might have an influence on the Ownership and the Location components.
While the results from the prosthesis user are anecdotal in nature, they serve to underscore and emphasize the statistical findings observed in able-bodied subjects. Note that these results often highlight apparent differences among experimental conditions, both concerning the number of sensors and the implemented delay in the myoelectric controller. Despite preliminary, these outcomes indicate the potential impact that sEMG interfaces may have on the embodiment of a virtual prosthesis.
These results underscore the intimate connection of this experiment design with the subcomponent agency, illustrating how the chosen embodiment test, along with testing conditions and/or protocol, strongly influences a portion of the overall embodiment experience for the prosthesis user.
Results from TAC demonstrated functional enhancements associated with an increased number of sEMG sensors employed. In fact, as we hypothesized, we obtain significantly higher SR and lower TtT values for condition HD compared to LD. These results align with existing literature on myoelectric control [49]. Moreover, when combined with our significant results on subjective embodiment, which also relates to the number of sensors, it is reasonable to assume that augmented functionality contributes to the sense of embodiment in a VR environment. Although anecdotal, Fig.Ā 10 also demonstrates an increasing subjective embodiment with higher functionalities irrespective of the sensor configuration (LD or HD), and with larger effect for HD sensors which supports our hypothesis H4. This assertion is further supported by [6], which emphasizes the strong association between agency, and volition. Furthermore, we observed an immediate impact of VR training, as evidenced by improved time-to-target performance (Fig.Ā 9b, e, and h). Note that participants already experienced VR for 10Ā min during the embodiment phase of the experimental protocol, followed by a 10-minute training during the functional phase.
Conclusion
In this study, we investigated the impact of different sEMG configurations and delay on the sense of embodiment and functional outcomes in an immersive virtual reality setup. Results from the functional assessment demonstrated that higher success rates are achieved by employing a higher number of sEMG sensors (named HD condition). Likewise, results from the questionnaire shows enhanced subjective embodiment during the HD condition. Albeit no significant differences among conditions, the virtual arm proprioceptive error (VPE) for LD sEMG sensors achieved smaller values in the synchronous condition compared to the asynchronous condition. Even if the performance of our myoelectric control is in alignment with the state of the art, the absence of significant changes in response to delayed control highlights the persisting limitations in developing simultaneous and continuous myoelectric control solutions. While this study has been conducted using a regression algorithm, future studies will investigate the effect of different myoelectric control methods and their impact in the Sense of Embodiment. Regarding the use of virtual reality environments as a testing platform, all functional metrics showed an immediate functional improvement (without significance) after training in VR environments. Nevertheless, the non-individualized anthropometry dimensions of our VR environment may reduce the general embodied experience and in particular, any location-based metric. Therefore, the adaptation of the proprioceptive drift experiment into immersive VR environments should be further explored. In summary, the combination of functional metrics with subjective embodiment scores provides a preliminary outcome suggesting the positive relationship between both aspects. Limiting our experimentation to able-bodied subjects overcomes issues with the varied and pathological muscle conditions exhibited by the population of prosthesis users. While this approach also facilitates the inclusion of a greater number of participants for statistical analysis, it comes at the expense of restricting the applicability of our conclusions to the target population. Nevertheless, we conducted the same experimental protocol with a prosthesis user, aiming to investigate the potential generalization of our results to real end-users. Recent promising results of machine learning myoelectric control algorithms, along with their incorporation into commercial systems, emphasize the importance of extending this evaluation to a larger group of participants with limb loss. The study shows the significance of sEMG interfaces into both function and sense of embodiment in immersive virtual reality. Results on able-bodied subjects reveal a positive relationship between performance and subjective embodiment, calling for further investigations to advance the natural integration of prostheses.
Availability of data and materials
All relevant data is within the paper. Detailed information may be available from the corresponding author after request.
Change history
27 April 2024
A Correction to this paper has been published: https://doi.org/10.1186/s12984-024-01368-z
References
Cordella F, Ciancio AL, Sacchetti R, Davalli A, Cutti AG, Guglielmelli E, et al. Literature review on needs of upper limb prosthesis users. Front Neurosci. 2016;10:209.
Mendez V, Iberite F, Shokur S, Micera S. Current solutions and future trends for robotic prosthetic hands. Annu Rev Control Robot Auton Syst. 2021;4:595ā627.
Bekrater-Bodmann R. Factors associated with prosthesis embodiment and its importance for prosthetic satisfaction in lower limb amputees. Front Neurorobot. 2021;14: 604376.
Gouzien A, De Vignemont F, Touillet A, Martinet N, De Graaf J, Jarrasse N, et al. Reachability and the sense of embodiment in amputees using prostheses. Sci Rep. 2017;7(1):1ā10.
Farina D, Vujaklija I, BrĆ„nemark R, Bull AM, Dietl H, Graimann B, etĀ al. Toward higher-performance bionic limbs for wider clinical use. Nat Biomed Eng. 2021:1-13.
Zbinden J, Lendaro E, Ortiz-Catalan M. Prosthetic embodiment: systematic review on definitions, measures, and experimental paradigms. J Neuroeng Rehabil. 2022;19(1):37.
Zbinden J, Lendaro E, Ortiz-Catalan M. A multi-dimensional framework for prosthetic embodiment: a perspective for translational research. J Neuroeng Rehabil. 2022;19(1):122.
Sensinger JW, Dosen S. A review of sensory feedback in upper-limb prostheses from the perspective of human motor control. Front Neurosci. 2020;14:345.
Scheme E, Englehart K. Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J Rehabil Res Dev. 2011;48(6).
Castellini C, Van Der Smagt P. Surface EMG in advanced hand prosthetics. Biol Cybern. 2009;100:35ā47.
Muceli S, Farina D. Simultaneous and proportional estimation of hand kinematics from EMG during mirrored movements at multiple degrees-of-freedom. IEEE Trans Neural Syst Rehabil Eng. 2011;20(3):371ā8.
Merletti R, Holobar A, Farina D. Analysis of motor units with high-density surface electromyography. J Electromyogr Kinesiol. 2008;18(6):879ā90.
Capsi-Morales P, Piazza C, Catalano MG, Grioli G, Schiavon L, Fiaschi E, et al. Comparison between rigid and soft poly-articulated prosthetic hands in non-expert myo-electric users shows advantages of soft robotics. Sci Rep. 2021;11(1):1ā15.
Terlaak B, Bouwsema H, van der Sluis CK, Bongers RM. Virtual training of the myosignal. PLoS ONE. 2015;10(9): e0137161.
Woodward RB, Hargrove LJ. Adapting myoelectric control in real-time using a virtual environment. J Neuroeng Rehabil. 2019;16(1):1ā12.
Murray CD. Embodiment and prosthetics. In: Psychoprosthetics. Springer; 2008. p. 119ā29.
Maimon-Mor RO, Makin TR. Is an artificial limb embodied as a hand? Brain decoding in prosthetic limb users. PLoS Biol. 2020;18(6): e3000729.
Page DM, George JA, Kluger DT, Duncan C, Wendelken S, Davis T, et al. Motor control and sensory feedback enhance prosthesis embodiment and reduce phantom pain after long-term hand amputation. Front Hum Neurosci. 2018;12:352.
Lopez C, Halje P, Blanke O. Body ownership and embodiment: vestibular and multisensory mechanisms. Neurophysiol Clin. 2008;38:149ā61.
Arzy S, Thut G, Mohr C, Michel CM, Blanke O. Neural basis of embodiment: distinct contributions of temporoparietal junction and extrastriate body area. J Neurosci. 2006;26(31):8074ā81.
Newport R, Preston C. Pulling the finger off disrupts agency, embodiment and peripersonal space. Perception. 2010;39(9):1296ā8 (PMID: 21125957).
Kilteni K, Groten R, Slater M. The sense of embodiment in virtual reality. Presence Teleoperators Virtual Environ. 2012;21(4):373ā87.
Beckerle P, Castellini C, Lenggenhager B. Robotic interfaces for cognitive psychology and embodiment research: a research roadmap. WIREs Cognitive Science. 2019;10(2).
Christ O, Reiner M. Perspectives and possible applications of the rubber hand and virtual hand illusion in non-invasive rehabilitation: technological improvements and their consequences. Neurosci Biobehav Rev. 2014;44:33ā44.
Tsay A, Savage G, Allen TJ, Proske U. Limb position sense, proprioceptive drift and muscle thixotropy at the human elbow joint: proprioceptive drift at the human elbow joint. J Physiol. 2014;592(12):2679ā94.
Botvinick M, Cohen J. Rubber āfeelā touch that eyes see. Nature. 1998;391(6669):756ā6.
Frƶhner J, Salvietti G, Beckerle P, Prattichizzo D. Can wearable haptic devices foster the embodiment of virtual limbs? IEEE Trans Haptics. 2018;12(3):339ā49.
Fribourg R, Argelaguet F, Hoyet L, Lecuyer A. Studying the Sense of Embodiment in VR Shared Experiences. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE;. p. 273-80.
Bovet S, Debarba HG, Herbelin B, Molla E, Boulic R. The critical role of self-contact for embodiment in virtual reality. IEEE Trans Visual Comput Graphics. 2018;24(4):1428ā36.
Wenk N, Penalver-Andres J, Buetler KA, Nef T, MĆ¼ri RM, Marchal-Crespo L. Effect of immersive visualization technologies on cognitive load, motivation, usability, and embodiment. Virtual Reality. 2021.
Engdahl SM, Meehan SK, Gates DH. Differential experiences of embodiment between body-powered and myoelectric prosthesis users. Sci Rep. 2020;10(1):1ā10.
Frƶhner J, Beckerle P, Endo S, Hirche S. An embodiment paradigm in evaluation of human-in-the-loop control. IFAC-PapersOnLine. 2019;51(34):104ā9.
Simon AM, Hargrove LJ, Lock BA. Kuiken TA. The Target Achievement Control Test: evaluating real-time myoelectric pattern recognition control of a multifunctional upper-limb prosthesis; 2014. p. 17.
Longo MR, SchĆ¼Ć¼r F, Kammers MPM, Tsakiris M, Haggard P. What is embodiment? A psychometric approach. Cognition. 2008;107(3):978ā98.
Nowak M, Vujaklija I, Sturma A, Castellini C, Farina D. Simultaneous and proportional real-time myocontrol of up to three degrees of freedom of the wrist and hand. IEEE Trans Biomed Eng. 2022.
Smith LH, Hargrove LJ, Lock BA, Kuiken TA. Determining the optimal window length for pattern recognition-based myoelectric control: balancing the competing effects of classification error and controller delay. IEEE Trans Neural Syst Rehabil Eng. 2011;19(2):186ā92.
Gijsberts A, Bohra R, SierraĀ GonzĆ”lez D, Werner A, Nowak M, Caputo B, etĀ al. Stable myoelectric control of a hand prosthesis using non-linear incremental learning. Front Neurorobotics. 2014:8.
Strazzulla I, Nowak M, Controzzi M, Cipriani C, Castellini C. Online bimanual manipulation using surface electromyography and incremental learning. IEEE Trans Neural Syst Rehabil Eng. 2016;25(3):227ā34.
Nissler C, Nowak M, Connan M, BĆ¼ttner S, Vogel J, Kossyk I, et al. Vita-an everyday virtual reality setup for prosthetics and upper-limb rehabilitation. J Neural Eng. 2019;16(2): 026039.
Jaeger L, Baptista RS, Basla C, Capsi-Morales P, Kim YK, Nakajima S, et al. How the CYBATHLON competition has advanced assistive technologies. Annu Rev Control Robot Auton Syst. 2023;6:447ā76.
Tsakiris M, Haggard P. The rubber hand illusion revisited: visuotactile integration and self-attribution. J Exp Psychol Hum Percept Perform. 2005;31(1):80.
Kammers MP, de Vignemont F, Verhagen L, Dijkerman HC. The rubber hand illusion in action. Neuropsychologia. 2009;47(1):204ā11.
Davies AMA, White RC, Thew G, Aimola NM, Davies M. Visual capture of action, experience of ownership, and the illusion of self-touch: a new rubber hand paradigm. Perception. 2010;39(6):830ā8.
Jsselsteijn WA, de Kort YAW, Haans A. Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence Teleoperators Virtual Environ. 2006;15(4):455ā64.
Kammers MPM, de Vignemont F, Verhagen L, Dijkerman HC. The rubber hand illusion in action. Neuropsychologia. 2009;47(1):204ā11.
Rohde M, Di Luca M, Ernst MO. The rubber hand illusion: feeling of ownership and proprioceptive drift do not go hand in hand. PLOS ONE. 2011;6(6):1ā9.
Yuan Y, Steed A, Is the rubber hand illusion induced by immersive virtual reality? In IEEE Virtual Reality Conference (VR). Boston, MA, USA: IEEE. 2010;2010:95ā102.
Romano D, Maravita A, Perugini M. Psychometric properties of the embodiment scale for the rubber hand illusion and its relation with individual differences. Sci Rep. 2021;11(1):5029.
Fougner A, Stavdahl O, Kyberd PJ, Losier YG, Parker PA. Control of upper limb prostheses: terminology and proportional myoelectric control-a review. IEEE Trans Neural Syst Rehabil Eng. 2012;20(5):663ā77.
Acknowledgements
The authors would like to thank Raoul Bongers for his scientific guidance in the protocol design, and all the volunteers that participated in this study.
Funding
Open Access funding enabled and organized by Projekt DEAL. This work was supported by TUM AGENDA 2030, funded by the Federal Ministry of Education and Research (BMBF) and the Free State of Bavaria under the Excellence Strategy of the Federal Government and the Laender as well as by the Hightech Agenda Bavaria.
Author information
Authors and Affiliations
Contributions
TSC, PC and CP were primarily responsible for the design and development of the virtual reality environments and to conduct the experiments. MC and CC were primarily responsible for the control algorithm. CP, CC and PB were primarily responsible for the design of the experimental protocol. All authors contributed and approved the manuscript
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The following study was approved by the Ethics Commission of the Technical University of Munich, and all participants gave their informed consent.
Consent for publication
Subjects have given consent to the publication of the data present in this manuscript.
Competing interests
The authors declare that they have no Conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised: Figure 2 citation under the sub-heading EMG setup and data collection has been updated.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
CastaƱeda, T.S., Connan, M., Capsi-Morales, P. et al. Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses. J NeuroEngineering Rehabil 21, 57 (2024). https://doi.org/10.1186/s12984-024-01352-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12984-024-01352-7