The influence of scaffolding on intrinsic motivation and autonomous adherence to a game-based, sparsely supervised home rehabilitation program for people with upper extremity hemiparesis due to stroke. A randomized controlled trial

Abstract

Background

This parallel, randomized controlled trial examined intrinsic motivation, adherence and motor function improvement demonstrated by two groups of subjects that performed a 12-week, home-based upper extremity rehabilitation program. Seventeen subjects played scaffolded games, presenting eight to twelve discrete levels of increasing difficulty. Sixteen subjects performed the same activities controlled by success algorithms that modify game difficulty incrementally.

Methods

33 persons 20–80 years of age, at least 6 months post stroke with moderate to mild hemiparesis were randomized using a random number generator into the two groups. They were tested using the Action Research Arm Test, Upper Extremity Fugl Meyer Assessment, Stroke Impact Scale and Intrinsic Motivation Inventory pre and post training. Adherence was measured using timestamps generated by the gaming system. Subjects had the Home Virtual Rehabilitation System (Qiu in J Neuroeng Rehabil 17: 1–10, 2020) placed in their homes and were taught to perform rehabilitation games using it. Subjects were instructed to train twenty minutes per day but were allowed to train as much as they chose. Subjects trained for 12 weeks without appointments and received intermittent support from study staff. Group outcomes were compared using ANOVA. Correlations between subject demographics and adherence, as well as motor outcome, were evaluated using Pearson Correlation Coefficients.

Results

There were 5 dropouts and no adverse events. The main effect of time was statistically significant for four of the five clinical outcome measures. There were no significant training group by time interactions. Measures of adherence did not differ significantly between groups. The combined groups improved their UEFMA scores on average by 5.85 (95% CI 4.73–6.98). 21 subjects from both groups demonstrating improvements in UEFMA scores of at least 5 points, exceeding the minimal clinically important difference of 4.25. IMI scores were stable pre to post training.

Conclusions

Scaffolding challenges during game based rehabilitation did not elicit higher levels of adherence when compared to algorithm control of game difficulty. Both sparsely supervised programs of game-based treatment in the home were sufficient to elicit statistically significant, clinically meaningful improvements in motor function and activities of daily living.

Trial registration: Clinical Trials.gov—NCT03985761, Registered June 14, 2019.

Introduction

Despite decades of research attempting to remediate upper extremity impairments following stroke, a rehabilitation approach that elicits substantial improvements in function that do not decay over time has not been developed [2]. This points to a need for opportunities for persons with residual impairments following stroke to work on their arm and hand function away from the clinical environment with relative independence [3]. The use of traditional and technology-supported home-based rehabilitation programs has increased steadily in the last two decades and was further accelerated by the COVID–19 pandemic [4]. Short term and directly supervised telerehabilitation programs produce outcomes comparable to those of clinic-based treatments [5, 6]. Longer programs and sparsely supervised programs have not been studied as well, and outcomes are less consistent. In general, adherence to programs of activity designed to improve or maintain motor function following a stroke is relatively low [7]. Multiple barriers to consistent performance of motor function training activities exist, including low motivation as well as a lack of interest in, or enjoyment of, training activities [8]. Multiple authors have proposed that game-based rehabilitation activities may help overcome these barriers and provide a solution to low adherence to home based rehabilitation programs [9,10,11]. This said, the published evidence presents a range of adherence rates to gamified, home based rehabilitation, suggesting that simply presenting a rehabilitation activity as a game might not result in across the board improvements in adherence [9, 12,13,14,15,16,17]. Multiple factors have been identified as possible causes for varied adherence to technology supported rehabilitation interventions in the home [9, 18, 19].

The gaming industry utilizes a wide variety of gaming mechanics, processes that govern the way a game flows, information is presented, and player success or failure is communicated. This influences the frequency with which players pick up a game and play it, as well as the amount of time they play a game after initiating [20]. This study focused on scaffolding, a very common gaming mechanism that presents a relatively easy version of a game, followed by gradually ascending levels of difficulty as a participant succeeds [21]. This affords the participant immediate initial feelings of self-efficacy and then proceeds to challenge them. Appropriate levels of challenge [22] and feelings of self-efficacy [23] are both associated with higher levels of motivation, as is the clear knowledge of results feedback [22] a participant receives when they are presented with a new challenge after they succeed or they are required to repeat a level if they fail.

This study utilized a parallel randomized clinical trial to examine the adherence levels of subjects with stroke performing a 12-week, home-based upper extremity rehabilitation program incorporating simulations that used scaffolding to that of a control group of subjects that performed the same activities controlled by success algorithms that increase and decrease game difficulty incrementally and undetectably [24, 25]. Our overall study question focused on autonomous adherence to the training program by setting the subjects up with the system and having them perform their training without direct supervision or appointments. The primary analysis focused on the impact of scaffolding on adherence, by tracking total treatment time using system-collected measurement of actual game play frequency. Secondary analyses examined (1) the effect of scaffolding on motivation by analyzing pre and post training Intrinsic Motivation Inventory scores and (2) the effectiveness of the training programs using clinical measures of upper extremity function and self-reported measures of hand function and activities of daily living.

Methods

This protocol was approved by the Institutional Review Boards of the New Jersey Institute of Technology and Rutgers, The State University of New Jersey in accordance with the Declaration of Helsinki. The protocol was registered at Clinical Trails.gov NCT03985761 on June 14, 2019.

Subjects: Inclusion criteria were: (a) 20–80 years old, (b) diagnosis of stroke confirmed from medical records, (c) score greater than or equal to 22 on the Montreal Cognitive Assessment [26], (d) visual field perception that allowed for attention to an entire 24″ computer screen, (e) proprioception sufficient to perform training activities without looking at their hand, (f) Upper Extremity Fugl-Meyer Assessment (UEFMA) score of 10–60/66 [27] and (g) receptive and expressive communication consistent with informed consent, and h) at least 6 months post stroke. Exclusion criteria were: (a) upper extremity orthopedic dysfunction that would limit upper extremity activity, and (b) chronic central nervous system pathology other than stroke. Subjects were recruited via local clinician referral and at stroke support groups. Subjects were screened and consented by a study coordinator. After this they were assigned to one of either the Enhanced Motivation—scaffolding (EM) or Algorithm Controlled (AC) group using a random number generator (https://www.random.org/), following a simple randomization pattern. Subjects were blinded to treatment group allocation and the comparison being examined.

Training system

The Home Virtual Rehabilitation System (HoVRS) is a computer based rehabilitation system designed to support independent training as well as remotely supervised training in the homes of persons with stroke (please see [1] for a detailed description of the system). HoVRS consists of two subsystems: (1) a patient-based system that presents rehabilitation games and (2) a cloud-based online data pipeline that allows for asynchronous monitoring and remote supervision. The patient-based system utilizes arm, forearm, wrist and hand position data collected by a Leap Motion Controller^™ (LMC), an infrared camera-based tracking device. Images collected by the cameras are transmitted using the LMC’s tracking software, which transforms the images into three dimensional representations. The LMC’s application programming interface estimates relative wrist, finger and finger positions, allowing the system to train specific motions of the fingers (flexion, extension and individuation) forearm (pronation and supination), and wrist (flexion, extension, radial and ulnar deviation). Tracking of hand position in 3D space allows for training of all elbow and shoulder movements as well. Upper extremity movements are used to control game play in a suite of games developed in the Unity 3D^™ game engine. A variety of support systems, including mechanical arm supports and tabletop forearm platforms, were utilized as needed to maintain a participant’s hand in the active workspace of the LMC during arm, wrist or finger activities. Software consists of a library of twelve games, designed by our team to train shoulder/elbow, forearm, wrist and finger motions. Basic games train movements in isolation, while more advanced games train coordinated combinations of movements. Games are designed to accommodate a wide variety of active movement abilities via a calibration protocol that measures and scales the amount of patient movement required to elicit avatar movement in the games. Game speeds, target/obstacle densities and sensory presentations are also scaled using the approaches described below to accommodate patients with moderate to severe impairments and challenge them as they progress.

Treatment groups

The enhanced motivation (EM) group played two to five of the twelve available rehabilitation games, depending on their goals and the movements they wanted to train. These games provided the user with eight to twelve levels of gradually increasing difficulty and complexity (scaffolding). A screen announced each level change and the graphics for each new level changed substantially. Scoring opportunities increased at each new level as well.

There was a wide variety of methods for increasing EM game difficulty. For example, the car simulation trained a single movement for the first two levels (1) hand opening and closing to speed up and slow down the car to manage speed bumps without crashing and then (2) pronating and supinating to change lanes to avoid cones). For the third level the two movements / obstacles were combined. After this, the fourth level added a pace car in an adjacent lane that needed to be beaten to the end of the level or the game started over from the beginning. The fifth level added coins that could be collected to score additional points, while avoiding the cones, managing speed bumps and continuing to beat the pace car. Each subsequent level involved increasing the proximity of cones to coins (requiring more precise movements) decreasing the speed needed to manage a bump without crashing (requiring more complete and faster hand closing) or increasing the speed of the pace car. In a similar fashion, the first level of the maze game presented a maze with wide rooms, few turns and very few wrong turns before reaching the ladder to the next level. Targets were collected to increase the score. Subsequent levels added more complex mazes, moving obstacles, narrower passages, edges that could be fallen off of requiring a return to the first level and multiple incorrect paths that needed to be retraced if the participant chose it. Simpler EM games were progressed by increasing game physics variables such as speed in a flying game or paddle size in a pong type game. These variables were changed abruptly at each level change. Variable changes rotated. For example, decreases in paddle size and ball speed alternated in the Padel game. Each EM game was scaled in a similar fashion, with abrupt changes in game difficulty and announced levels. EM formatted games advanced automatically after a participant successfully completed a level.

The algorithm control (AC) group also played two to five of the same twelve rehabilitation games. Game difficulty was modified using adaptive algorithms based on maintaining an eighty percent success rate over any given period of 60 s. Difficulty changes focused on a single element of difficulty and were designed to be incremental with the goal of making them imperceptible to subjects. Scoring opportunities and graphics did not change when the algorithms changed difficulty. For example, the AC version of the car game featured a process generated, random course of speed bumps and cones which subjects negotiated using both motions (hand opening / closing and pronation/supination) that went on forever. The algorithm adjusted difficulty by increasing or decreasing the speed of the pace car. The AC version of the maze simulation was scaled in a similar fashion. Maze training started at a medium level of complexity (walls and dead ends) that did not change. Runner speed increased when the subject captured at least 80 percent of targets. Simple AC games like the pong or flying games changed via algorithm, with no announcement of difficulty changes. The specific parameter (speed, obstacle density, paddle size, etc.) was designated at follow up sessions with the study therapist or technologist.

Protocol followed by subjects in both groups: After randomization to one of the two interventions, subjects used the New Jersey Institute of Technology (NJIT) -HoVRS system to train movement of their shoulder, elbow, forearm, wrist, and fingers (Please see a detailed description of the HoVRS system in Qiu et al. 2021 [1]). Study teams consisting of a Physical Therapist and a technologist, who were not blinded to group allocation, set up the apparatus with all subjects in their homes at an initial visit and trained them to set up the system, open their assigned rehabilitation games, and play them.

Initially, subjects from both groups were assigned three simple simulations: one each for the shoulder / elbow, forearm, wrist, and fingers. Subjects were assigned the EM or AC versions of the same games which targeted movement patterns that limited their ability to perform daily functional tasks as determined by the study therapist during pre-testing. (Please see Appendix One for a description of the rehabilitation games.) At every follow up session (in person or on line), the study therapist updated the subjects’ training routines. Individual games were adjusted as needed by increasing the amount of movement required to affect game play. For all subjects, when simple games were mastered, games that combined forearm or wrist movement with finger movements (e.g. combining hand opening and pronation / supination) or games that combined finger movement with hand transport (e.g. moving the hand across a piano keyboard to press specific keys) were introduced. Subjects played the rehabilitation games in their homes independently, with on-line or in-person support as needed. All subjects were encouraged to play at least twenty minutes daily, but were allowed to play the games as much as they liked.

Data collection

All data were collected in subjects’ homes.

Outcome measures

Total treatment time and the frequency of training was used to evaluate adherence to the training programs. Both were monitored and measured by tracking performance data collected by the system. Total treatment time over the 12-week training period was estimated for each subject using computer timestamps of the files with performance data saved after each training session. In addition, the number of training sessions over the 12-week training period was evaluated.

The impact of scaffolding on motivation was measured using the Intrinsic Motivation Inventory (IMI) [28]. Subjects completed a twelve-item version of the Intrinsic Motivation Interview (See Appendix 1) after the first and last training weeks to evaluate the impact of training game configuration on motivation to play the games, and the impact of extended play of the games (12 weeks) on motivation.

To measure the impact of training on changes in upper extremity motor function, subjects completed the UEFMA [27], and Action Research Arm Test (ARAT) [29], just prior to and immediately after their participation in training. In addition, subjects completed the Hand, Activities of Daily Living, and Participation sub-scales of the Stroke Impact Scale (SIS) [30]. Tests were administered by a trained therapist blinded to group assignment.

Data analysis

Primary and secondary analyses

Anderson–Darling normality test was used to check for baseline data normality. Total treatment time (adherence), the primary analysis, was not normally distributed and thus analyzed using Mann–Whitney U tests for between group comparisons and Wilcoxon signed‐ranks test for related samples. Secondary outcome measures were IMI, UEFMA, ARAT and SIS scores (all normally distributed). A one-between, one-within repeated measures ANOVA was used to examine the effects of the treatment group (Enhanced Motivation, Algorithm Controlled) and testing time (Baseline, Post) on the secondary outcome measures.

Correlations between baseline demographics, clinical measures and training adherence were evaluated using Pearson Correlation Coefficients for normally distributed data and Spearman Correlation Coefficients for abnormally distributed data. All analyses were performed in Minitab 22.

Results

Subjects

A total of 33 subjects (24 male and 9 female) satisfied the inclusion and exclusion criteria. Subject mean age was 57 (SD = 13). Mean time since stroke was 47 months (SD = 65) and baseline UEFMA was 43 (SD = 13). Subjects were randomized into EM (n = 17) and AC (n = 16) groups after baseline testing. The EM group had two dropouts, and the AC group had three. One of the AC group dropouts did not enjoy the games. The other four dropouts reported difficulties with setup and playing the games as reasons for discontinuing training. Dropouts did not differ substantially from non-dropouts in age (Mean = 57.8, SD = 12.8), time since CVA in months (Mean = 8, SD = 2.5), or baseline UEFMA score (Mean = 46.6, SD = 9.9). For the remaining subjects, there were no statistically significant differences in baseline characteristics between EM and AC groups (Please see Fig. 1 and Table 1).

Fig. 1

CONSORT diagram

Table 1 Baseline demographic and clinical test scores

In person follow up visits for the 12-week program ranged between one and six visits per subject for the study. The most common reason for an in-person visit was the need to replace simpler games with more challenging games. Only four subjects required an in person visit to address technical issues. Calls for technical support during the 12-week intervention ranged between zero and eleven. All but four subjects required five or less calls. The most common technical issues were related to internet provider issues, operating system updates and camera issues.

Adherence

Subjects that completed the protocol from both groups demonstrated substantial variance in adherence to the training protocol (total training time and number of training sessions). EM group subjects' training time ranged between 299 and 2672 min of training with a median training time of 966 (IQR = 442–1570) minutes. AC group subjects' training time ranged between 165 and 1208 min of training with a median training time of 680 (IQR = 412–902) minutes. This difference was not statistically significant (W = 159, p = 0.182). The within group variance and between group differences in the number of training sessions were smaller than those of total minutes. EM group subjects performed between 18 and 77 sessions. Mean number of sessions for the EM group was 48 (SD = 16). AC group subjects performed between 6 and 68 sessions. Mean number of sessions for the AC group was 37 (SD = 18). This difference was not statistically significant (F (1,26) = 2.56, p = 0.122). (See Table 2 and Fig. 2). There were no statistically significant correlations between demographic factors (age, time since stroke) or baseline impairment (UEFMA score) and adherence (See Table 3).

Table 2 Outcome measure scores

Fig. 2

Boxplots describing total training timed and total number of sessions

Table 3 Correlations between participant characteristics and IMI scores with adherence measures

Intrinsic motivation inventory

The main effect of time was statistically significant (F (1,26) = 7.83, p = 0.007), and positive, suggesting that extended play of the rehabilitation games did not result in a decrease in intrinsic motivation. There were no statistically significant between group differences in IMI scores at baseline or post intervention testing, and there was no statistically significant training group by time interaction (See Table 2). There were weak to moderate correlations between baseline as well as post intervention IMI and total training minutes, suggesting that there was a relationship between intrinsic motivation related to game play and adherence to the training protocol (See Table 3).

Clinical outcome measures

Both groups demonstrated improvements in motor function as evidenced by the fact that main effect of time was statistically significant for UEFMA (F (1,26) = 112.4, p < 0.001), ARAT (F (1,26) = 29.1, p < 0.001), SIS-ADL (F (1,28) = 26.2, p < 0.001), and SIS-Hand (F (1,26) = 5.7, p = 0.025). Subjects’ SIS—Participation scores did not change from pre to post-test. EM participants and AC participants demonstrated statistically significant improvements in UEFMA, ARAT and SIS-ADL. AC group participants also demonstrated statistically significant improvements in SIS-Hand, and SIS-Participation (Please see Table 2). There were no statistically significant training group by time interactions for any of the clinical outcome measures. Subjects in both groups improved their UEFMA scores on average by 5.85 (95% CI = 4.73—6.98). 12 of the 15 subjects in the EM group and 9 of the 14 subjects in the AC group demonstrated improvements in UEFMA score of at least 5 points that exceeded the published minimum clinically important difference (MCID) of 4.25 points, which was associated with meaningful improvements in a sample of persons with chronic stroke [31]. 4 of the 15 subjects in the EM group and 7 of the 14 subjects in the AC group demonstrated improvements in ARAT score that exceeded the published MCID for persons with chronic stroke [32]. There were no statistically significant correlations between training time and any of the clinical outcome measures (See Table 4).

Table 4 Correlations between adherence measures and clinical improvements

There were no statistically significant correlations between training time or number of training sessions and any clinical outcome measures (See Table 4). Pretest UEFMA score demonstrated negligible correlations with ARAT change (PCC = 0.055), SIS Hand (PCC = 0.190), and SIS ADL change (PCC = 0.082) and a weak, negative correlation (PCC = −0.249) with SIS Participation Change. The only statistically significant correlation (PCC = −0.614, p = 0.013) was a moderate, negative correlation between pre-intervention UEFMA and UEFMA improvement (See Fig. 3).

Fig. 3

Relationship between baseline motor impairment as measured by UEFMA score and improvements in UEFMA score. Red line = regression. Dashed blue line = the published minimum clinically important difference for persons with stroke [31]

Discussion

This study examined the adherence of a group of persons with upper extremity hemiparesis due to stroke who performed one of two different game-based, autonomous training programs targeting their paretic arms, hands and fingers. It is the first study examining a specific approach to game design focused on improving motivation and adherence to a home based training program in subjects without treatment appointments or extensive supervision. The lack of adherence levels or motor/functional outcomes that clearly favor the EM group, suggests that scaffolding alone may not provide a sufficient motivational stimulus to optimize these constructs. Similar functional/motor outcomes may also be related to the common elements of the two training programs (regular calibration of movement demands of the games, similar progressions from simple to more complex games, prescription of specific games based on the same guidelines and overlap between game parameters scaled). We feel that this points to the need for continued study of this area of inquiry to investigate the additive effects of other motivational enhancement techniques that might include competition, cooperative play or narrative. There were no statistically significant correlations between training time and improvements in clinical outcomes. The lack of a relationship between training time and outcome differs from some studies of the relationship between UE rehabilitation time and outcome [33] but is similar to other studies that cite a relatively weak relationship between training dosage and clinical outcomes after a minimum training threshold is achieved [34, 35].

Overall adherence to both training programs was modest. Dropout rates were 11% and 13% for the two groups and total training time was lower than that of many studies of home-based rehabilitation. This is likely due to the fact that (1) the intervention was relatively long (12 weeks), and (2) subjects did not have to train by appointment. When comparing subjects in studies examining sparsely supervised, home based rehabilitation interventions, adherence rates and training time were better than those of subjects in a study by Standen [17] but not as good as those in a study by Rand [9]. This said, UEFMA, ARAT, and self-reported ADL improvements in this study were consistent and strong, suggesting that persons with stroke can train productively in their homes with minimal supervision. Designing home-based rehabilitation systems that can be utilized without constant direct supervision of a therapist holds the potential to expand access to economically disadvantaged and underserved populations. We feel that the success of the subjects in this trial would justify further studies of game-based, home-based interventions with sparsely supervised participants.

The two programs differed in the level of explicit feedback related to success that they were provided during game play. The EM group, which was presented with more explicit feedback, demonstrated similar IMI scores immediately after the first week of training and immediately after the last week of training compared to the AC group that was provided less explicit feedback. Despite this similarity and the fact that there was a moderate correlation between IMI scores and total training time, the EM group demonstrated slightly larger median training times over the 12-week training program than the AC group. While this difference was not statistically significant, this suggests that there was some aspect of the interaction between the two training programs and subjects that differed, which was not captured by the IMI. The relatively high IMI scores and statistically significant increase in total IMI score over time might suggest that both of the training programs were relatively engaging over the course of training Engagement in training has been cited as a factor influencing experience dependent neuroplasticity that underlies motor recovery post-stroke [36]. This might suggest that high levels of interest and engagement in training might be an important variable related to the consistent improvements in motor function in spite of relatively modest total training volumes.

Advanced age, chronicity of strokes and high levels of motor impairment have been cited as potential barriers to the use of and ability to benefit from technology supported rehabilitation approaches [37]. Interestingly, our data did not support these generalizations, as evidenced by the lack of correlation between adherence and these variables. An important design objective for this system was the ability to accommodate persons with significant upper extremity motor impairments. The fact that the correlation between baseline UEFMA score was negative, with more impaired subjects demonstrating larger improvements in motor function, would lead us to propose that the subjects best suited to benefit from sparsely supervised game-based training might be those with moderate upper extremity hemiparesis (UEFMA scores between 20 and 53). With these statements made, it is obvious that further study designed to evaluate these assertions prospectively, in a larger group of subjects, is indicated before definitive conclusions can be made.

Training time data in this study were abnormally distributed due to the small sample and positive outliers in the EM group. This may have led to the non-significant statistical comparison between EM and AC group training times despite the trend toward significance. A larger study will be necessary to evaluate this hypothesis definitively. Limitations of this study include the lack of retention testing of clinical outcomes. This said, we would argue that the fair outcomes and retention of gains demonstrated by persons with chronic hemiparesis in a wide variety of rehabilitation trials [38] presents support for the need lifetime motor practice [39]. Effective home training activities that can be performed with minimal supervision would be an important resource to help fill this need. Finally, our subjects volunteered to participate in a study of technology supported rehabilitation which might decrease our findings' generalizability to persons who are highly averse to technology.

Conclusions

This study examined the impact of scaffolding on adherence to a sparsely supervised home-based training program targeting the paretic upper extremity of persons with stroke. Scaffolding challenges during game based rehabilitation did not elicit higher levels of adherence, intrinsic motivation or motor improvement than a group that trained with algorithm control of game difficulty. Both sparsely supervised programs of game-based treatment in the home were sufficient to elicit statistically significant, clinically meaningful improvements in motor function and activities of daily living, which suggests that a sparsely supervised, game-based training program performed in the home can have meaningful, positive effects on arm, hand and finger function in persons with chronic hemiparesis due to stroke.

Games	Effector(s) trained	Movement pattern(s) trained
Soccer goalie	Shoulder	Arm elevation
Padel	Shoulder	Arm elevation
Maze	Shoulder & elbow	Horizontal plane shoulder and elbow dissociation
Fruit catch	Shoulder, elbow, forearm	Hand transport in horizontal plane, pronation and supination
Brick breaking	Forearm	Pronation
Brick breaking	Wrist	Radial and ulnar deviation
Desert pilot	Wrist	Extension
Urban aviator	Fingers	Finger extension
Auto racing	Forearm, fingers	Hand opening and closing, pronation and supination
Piano teacher	Shoulder, elbow, fingers	Finger individuation (stationary), Hand transport combined with finger individuation
Fruit pick	Shoulder, elbow, fingers	Hand transport combined with pincer grasp
Solitaire	Shoulder, elbow, fingers	Hand transport combined with pincer grasp – precision release