Experimental setup
The six DoF arm exoskeleton rehabilitation robot ARMin was employed for this experiment (Fig. 1) [39]. The actuated DoFs are: shoulder elevation, abduction/adduction, internal/external rotation, elbow flexion, forearm supination/pronation, and wrist extension/flexion. To measure the interaction forces between the human and robot, three force/torque sensors (Mini45, ATI Industrial Automation, USA) were located at the hand module and the upper and lower arm cuffs, where the participant’s arm is attached to the exoskeleton (Fig. 1). The motion control of ARMin and simulation of pendulum dynamics were performed with Simulink Realtime R2017b (MathWorks, Massachusetts, USA) at 3 kHz.
The Unity3D game engine (Unity Technologies, USA) was employed to implement the virtual environment (VE) and the experimental protocol. The VE was displayed on an LED screen (109 cm, 43UD79, LG, South Korea) located in front of the participant. The robot motion control and the game software communicated with UDP protocol.
Pendulum dynamics
The task to be learned consisted of inverting a virtual pendulum by moving the exoskeleton hand module. The position and orientation of the robot hand module were mapped to the virtual pendulum pivot point with an approximate scaling factor of 0.5. The virtual pendulum pivot point is depicted as a handle inside the black circle of the pendulum in Fig. 2.
Participants could move and rotate the hand module in 3D, but only the movements in the vertical plane affected the pendulum movement. The rotation of the hand module, although did not influence the pendulum movement, still matched with the virtual pendulum handle depicted within the pendulum pivot point (EE in Fig. 2) to facilitate the understanding of the robot-pendulum interface. While only the movement in the horizontal direction would be sufficient to invert the pendulum, we allowed movements in the vertical direction to analyze the effect of the arm weight support on the participants’ movements in the direction of gravity.
The pendulum dynamics had only one internal DoF: the pendulum angle (\(\theta\)). The pendulum motion was simulated according to the following equation:
$$\begin{aligned} \ddot{\theta } = - \frac{1}{l} \bigg ( \ddot{y} \cos \theta + (\ddot{z} + g) \sin \theta \bigg ) - \frac{c}{ml^2}{\dot{\theta }} \end{aligned}$$
(1)
where y and z are the horizontal and vertical directions, respectively. The pendulum mass (m) and rod length (l) were selected as 2.5 kg and 0.35 m (0.25 m for the transfer task). The damping coefficient (c= 0.16 N.s/rad) was required to stabilize the pendulum. We wanted to keep the velocity/acceleration-induced forces of the pendulum (e.g., centrifugal and inertial forces) high, so that the participants could better feel that the pendulum reacts to their actions. To do this, we could either increase the mass—making the pendulum heavier and thus, increasing excessively the participants’ fatigue and limiting the experiment duration—or reduce the gravity so that the pendulum is not too heavy while the mass is high. Therefore, we reduced the gravity coefficient (g) to 25% of the real earth gravity.
Haptic rendering
The haptic rendering forces of the pendulum in the direction of the pendulum rod were calculated according to the following equation:
$$\begin{aligned} F_{rod} = m \bigg ({\dot{\theta }}^2 l - \ddot{y} \sin \theta + (g+\ddot{z}) \cos \theta \bigg ). \end{aligned}$$
(2)
The \(F_{rod}\) was rendered at the robot hand module and transmitted to all robot joints using the hand module Jacobian. An invisible virtual safety table was also haptically rendered at the participants’ leg level to prevent collisions. A warning was presented on the screen every time participants touched the (invisible) table and were prompted to elevate the pendulum.
To reduce the variability in the joint null space and facilitate playing the game, the wrist flexion and shoulder internal rotation were fixed at 0\(^\circ\) and 30\(^\circ\) with position controllers. The position controllers were soft enough (\(\hbox {K}_p\) = 100 N.m.kg\(^{-1}\) rad\(^{-1}\), \(\hbox {K}_d\) = 20 N.m.s.kg\(^{-1}\) rad\(^{-1}\)) to make participants feel the haptic rendering forces of the pendulum in all arm joints.
A Disturbance Observer (DOB) was implemented for each robot joint to compensate for the robot disturbances (e.g., friction, robot weight) and allow the robot to follow the participants’ self-generated movements transparently [39]. The DOBs employed the force/torque measurements as input. However, please note that the perception of the pendulum haptic rendering was still slightly affected by the transparency of the robot as the robot inertia could not be fully compensated.
Arm weight support
We chose arm weight support as the assisting method because this type of assistance does not depend on the movement, and therefore, we expected minimal interaction between the supporting forces and the haptic rendering. If any interaction between the haptic rendering and the arm weight support was to be observed, this would most probably be more evident when the robot supports completely the weight of the participants’ arms, and therefore, we decided to support 100 \(\%\) of the participants’ arm weight.
The arm weight support method—first presented in [40]—uses a model of the participant’s arm to cancel the effect of gravity regardless of the arm pose. The method requires that the individual upper/lower arm weights are estimated. This estimation was performed at the beginning of the experiment, asking each participant to keep for 10 s a fixed pose [−10\(^\circ\) shoulder elevation, −10\(^\circ\) abduction, 30\(^\circ\) external rotation, −30\(^\circ\) elbow flexion, 0\(^\circ\) forearm supination, and 0\(^\circ\) wrist extension]. This fixed pose was selected to maximize the accuracy of the parameter estimation [40].
The inverted pendulum task
The experimental task consisted of inverting the pendulum and keeping it vertically inverted as long as possible (\(\theta = \pi\) rad, \({\dot{\theta }} =\) 0 rad/s) by moving the robot hand module. A score was shown on the screen as feedback to the participants. The score increased according to the following equation:
$$\begin{aligned} &\text {Score} = \int _{inv_{begin}}^{inv_{end}} k (\pi /6 -|\theta -\pi |) dt, \text { if:}\\&|\theta -\pi |< \pi /6 \text { rad, and } |{\dot{\theta }}| < \pi /3 \,\text {rad/s}. \end{aligned}$$
(3)
When the pendulum angle reached the inversion boundaries [\(|\theta -\pi | < \pi/6\)] rad with a small rotational speed (\(|{\dot{\theta }}| < \pi\)/3 rad/s), the inversion was considered as initiated (\(inv_{begin}\)), and the score started increasing proportionally (k = 19) to \(|\theta -\pi |\). The inversion was considered to end once the pendulum fell outside of these conditions (\(inv_{end}\)). Within each experimental block, if the participant dropped the pendulum down, the score was retained until it was inverted again. At the beginning of a new block, the score and \(\theta\) were reset to zero.
Study protocol
The experiment was approved by the Cantonal Ethics Committee and the Swiss Agency for Therapeutic Products (Swissmedics) and followed the Declaration of Helsinki. We recruited 41 healthy right-handed participants for the study—evaluated with the Waterloo handedness questionnaire [41]—, but due to a data acquisition problem with one participant, only 40 participants were included in the data analysis (20 females, 20 males, age mean: 29, std.: 5.7 y.o.). All participants gave written consent to take part in the experiment.
Participants were randomly assigned to one of four training modalities—ten participants per modality, between 4 and 6 females per modality. Each training modality corresponded to combinations of two factors: Haptic Rendering (HR) and arm Weight Support (WS):
-
Visual: Neither haptic rendering nor arm weight support was provided (HR:OFF, WS:OFF).
-
Supported Visual: Arm weight support was provided, but not haptic rendering (HR:OFF, WS:ON).
-
Visuo-Haptic: The haptic rendering of the pendulum dynamics was provided at the hand module, but not arm weight support (HR:ON, WS:OFF).
-
Supported Visuo-Haptic: Arm weight support was provided in addition to haptic rendering (HR:ON, WS:ON).
The experiment consisted of two experimental sessions that were one to three days apart. Participants sat comfortably on a chair with a backrest and their right arms were attached to the exoskeleton cuffs with Velcro® straps. The exoskeleton height and links lengths were adjusted for each participant. The experiment started with a short calibration phase (\(\approx\) 2–3 min) to adjust the height of the virtual safety table just above the participant’s legs and to estimate the individual participants’ upper/lower arm weights.
The participants were instructed to swing the pendulum, to invert it and keep it inverted as long as possible. The instructions could be read on the screen. We included an exemplary video to facilitate the task understanding. After the instructions, participants performed two baseline blocks (BL) of 30 s each with the Visuo-Haptic modality (Fig. 3). Visuo-Haptic was chosen as the test modality as it was closest to reality—i.e., with haptics and no support. The participants then performed two transfer baseline blocks (TBL) (30 s each) with the Visuo-Haptic modality but with a shorter pendulum. The shorter pendulum length—0.25 m instead of 0.35 m—corresponded to a higher pendulum natural (swing) frequency, therefore, resulted in different pendulum dynamics. Participants rested their arms for 30 s between all experimental blocks.
After baseline, the training phase started. The training consisted of 24 experimental blocks of 30 s each. Twenty of these blocks were training blocks (T), in which participants trained with the modality according to the group they were assigned—either Visual, Supported Visual, Visuo-Haptic, or Supported Visuo-Haptic. The other four blocks were catch-trial blocks (CT) in which the participants trained with the Visuo-Haptic modality (as in baseline). The CT blocks were included to detect and remove potential learning effects from the training blocks during data analysis. The order of the CT blocks was uniformly distributed over time—5th, 9th, 13th, and 17th blocks—to counterbalance potential learning effects when comparing the training and catch-trial blocks. Participants rested their arms for 30 s between all training blocks.
Shortly after the last training block, participants performed a short-term retention test. Similar to the baseline test blocks, the participants performed two short-term retention blocks (STR) and two transfer short-term retention blocks with the Visuo-Haptic modality. The first experimental session lasted around 1 h.
The second experimental session consisted of only the long-term retention test. Participants performed two long-term retention blocks (LTR) and two transfer long-term retention blocks with the Visuo-Haptic modality.
The participants’ sense of agency and subjective motivation were assessed with questionnaires (see Additional file 1: Questionnaire for a complete list) after the first two baseline blocks, right after the last training block, and after the first two long-term retention blocks. To assess the sense of agency, we adapted three statements from the embodiment questionnaire from Piryankova et al. [42] to the pendulum task. Participants ranked their agreement with each of the three statements using a Likert scale between -3 (“strongly disagree”) and 3 (“strongly agree”). Twelve statements from the well-established Intrinsic Motivation Inventory (IMI, [43]) were used to assess the participants’ motivation. These statements were selected from the four subscales: interest/enjoyment, effort/importance, pressure/tension, and perceived competence. A Likert scale between 1 (“not at all”) and 7 (“very true”) was used for the answers. The questionnaire was presented in English. The (previous) responses given during the experiment were always visible for each participant to minimize the possibility that differences in participants’ memory skills confound the results [44].
Outcome metrics
We used different metrics to analyze the participants’ task performance, movement, and physical effort. Each of these metrics provided one data point for one experimental block. Additionally, we used questionnaires to evaluate the participants’ level of agency and motivation.
Task performance
To evaluate how well participants performed the motor task—i.e., inverting the pendulum—the score was employed as the task performance metric. The score (eq. 3) increased depending on how vertically and still the pendulum was kept inverted, and for how long.
Movement strategy
To achieve a high score, one first needs to be able to lift the pendulum ball. This requires providing sufficient momentum to the pendulum ball. This can be achieved by making the pendulum pivot point lead the movement of the pendulum ball—i.e., when the pivot point applies a force to the ball through the rod.
The movement of the pivot point and the relative movement of the pendulum ball w.r.t. the pivot point are both almost cyclic. Thus, leading the pendulum ball corresponds to maintaining a positive phase difference between the pivot point movement and the relative movement of the pendulum ball w.r.t. the pivot point. To quantify this synchronicity, we first extracted the velocity of the pivot point and the relative velocity of the pendulum ball w.r.t. the pivot point in the horizontal direction, only during the time the pendulum was not yet inverted. We did not analyze the vertical direction because the analysis focuses on the part where the pendulum was not inverted, which ignores most of the data in the upward direction (\(+z\)), biasing the calculations. We then band-filtered the velocity time series to exclude the low-frequency drifts and high-frequency noise using second-order Butterworth bandpass filters with 0.5 Hz and 10 Hz cut-off frequencies. Next, the Hilbert transform was applied to the normalized velocity signals—with zero mean and unit standard deviation—and the phase signals (time-series) for the pivot point and the pendulum ball were obtained. Finally, the mean difference between the two phase signals within a block was calculated (movement phase difference). A positive movement phase difference is desired since it means the participant is leading the pendulum motion—i.e., the pivoting point is ahead of the pendulum ball. However, since the phase is defined only in the range between \(-\pi\) and \(\pi\), the optimum movement phase difference to achieve good performance is not known a priori, and needs to be checked with its correlation with task performance.
To evaluate how much participants explored the environment, we analyzed the participants’ movement variability with the standard deviation of the hand module position in both y—horizontal movement variability—and z—vertical movement variability—directions. Analyzing both directions independently allows for the evaluation of the potential direction-specific effects of weight support, which is acting only on the vertical direction, on the participants’ workspace variability.
Physical effort
The participants’ physical effort was estimated by norm averaging their joint torques. The generated joint torque was estimated by summing the recorded interaction torques and the torques needed by the participants to hold their arm at each position, online-calculated by the arm weight support algorithm [23].
Agency and motivation
The sense of agency and the four subscales of the IMI—i.e., interest/enjoyment, effort/importance, pressure/tension, perceived competence—had three questions each (Additional file 1: Questionnaire section). The averages of the three questions were calculated for each participant. Two participants did not answer one question each; therefore, the average of two questions was performed for the corresponding subscales.
Statistical analysis
To evaluate potential differences during baseline between training groups (factor: Visual, Supported Visual, Visuo-Haptic, Supported Visuo-Haptic), we compared the baseline data—mean of the two baseline blocks—for each outcome metric between groups using one-way ANOVA.
To evaluate whether the task performance (score) is associated with the movement phase difference between the pivoting point and the pendulum ball, the relationship between the movement phase difference and the score was analyzed with Pearson correlations, separately at BL, STR, and LTR, with each participant providing one data point.
To check if the learning continued linearly during the whole training or whether it reached a plateau by the end of the training (e.g., logarithmic), we fit generalized linear models for the score metric with block number as a continuous time variable similar to [45], and tested whether the addition of the nonlinear part significantly improved the model.
To determine how providing Haptic Rendering (HR) and/or Arm Weight Support (WS) affected the participants’ performance during training (T), compared to not providing HR/WS, we first subtracted the learning effects—mean of catch trials (CT)—from the training performance—mean of the 20 T blocks—for each outcome metric and participant, based on the assumption that learning curves were approximately linear. Then, the resulting T−CT values were analyzed with two-way ANOVAs (or two Kruskal-Wallis tests for the effects of HR and WS, respectively, if T−CT was non-normally distributed) with two factors: HR and WS; each factor having two levels: OFF and ON. If there was a significant (HR x WS) interaction, post hoc tests were performed to evaluate: the HR effect for WS:OFF and WS:ON; and the WS effect for HR:OFF and HR:ON.
To analyze the short-term learning effects of training with HR and/or WS, we took the difference of the short-term retention (STR) and baseline (BL) values—averaged between the two test blocks—of each outcome metric and participant. The effects of HR and WS on the STR−BL differences were analyzed either with two-way ANOVAs or with two Kruskal-Wallis tests, depending on the data distribution. For the analysis of the training modality on long-term learning, the same analyses were employed, but with the LTR−BL difference.
The short-/long-term effects of training with HR and/or WS on skill transfer were analyzed by comparing the differences between levels of HR and WS for the changes of score from baseline transfer to short-/long-term retention (STR/LTR) transfer using two-way ANOVAs. Furthermore, we analyzed if participants moved differently—i.e., with different movement variability and hand module speed—when performing the main task compared to the transfer task with a shorter pendulum rod at long-term retention. We used a mixed ANOVA for this comparison with pendulum length as a within-subject factor (levels: short, long), and training modality as the between-subjects factor.
The effect of HR and WS on participants’ agency and motivation levels after training (T−BL) and at long-term retention (LTR−BL) were analyzed using two-way ANOVA or two Kruskal-Wallis tests, depending on the questionnaire data distribution.
The normality of the data was visually inspected and evaluated using Kolmogorov-Smirnov tests from the Scipy module of Python. For one-/two-way ANOVAs, Afex package of R [46]; and for Kruskal-Wallis tests (used when the distributions were non-normal), Scipy module of Python [47] were employed. Bonferroni correction was used for multiple comparisons. The significance level was set to \(\alpha\) = 0.05 for all statistical tests.