Skip to main content

Technology Assisted Rehabilitation Patient Perception Questionnaire (TARPP-Q): development and implementation of an instrument to evaluate patients’ perception during training

Abstract

Background

The introduction of technology-assisted rehabilitation (TAR) uncovers promising challenges for the treatment of motor disorders, particularly if combined with exergaming. Patients with neurological diseases have proved to benefit from TAR, improving their performance in several activities. However, the subjective perception of the device has never been fully addressed, being a conditioning factor for its use. The aims of the study were: (a) to develop a questionnaire on patients’ personal experience with TAR and exergames in a real-world clinical setting; (b) to administer the questionnaire to a pilot group of neurologic patients to assess its feasibility and statistical properties.

Methods

A self-administrable and close-ended questionnaire, Technology Assisted Rehabilitation Patient Perception Questionnaire (TARPP-Q), designed by a multidisciplinary team, was developed in Italian through a Delphi procedure. An English translation has been developed with consensus, for understandability purposes. The ultimate version of the questionnaire was constituted of 10 questions (5 with multiple answers), totalling 29 items, exploring the patient’s performance and personal experience with TAR with Augmented Performance Feedback. TARPP-Q was then administered pre-post training in an observational, feasible, multi-centric study. The study involved in-patients aged between 18 and 85 with neurological diseases, admitted for rehabilitation with TAR (upper limb or gait). FIM scale was run to control functional performance.

Results

Forty-four patients were included in the study. All patients answered the TARPP-Q autonomously. There were no unaccounted answers. Exploratory factor analyses identified 4 factors: Positive attitude, Usability, Hindrance perception, and Distress. Internal consistency was measured at T0. The values of Cronbach’s alpha ranged from 0.72 (Distress) to 0.92 (Positive attitude). Functional Independence Measure (FIM®) scores and all TARPP-Q factors (Positive attitude, Usability, Hindrance perception, except for Distress (p = 0.11), significantly improved at the end of the treatment. A significant positive correlation between Positive attitude and Usability was also recorded.

Conclusions

The TARPP-Q highlights the importance of patients’ personal experience with TAR and exergaming. Large-scale applications of this questionnaire may clarify the role of patients’ perception of training effectiveness, helping to customize devices and interventions.

Introduction

Neuro-motor rehabilitation is defined as a problem-solving process focused on the betterment of a patient’s functional activities and aiming to improve both motor outcomes and quality of life [1]. Its effectiveness generally depends upon several heterogeneous factors, ranging from the training protocol to the patient’s characteristics [2, 3] and to external variables, such as the relationship between patient and health care professional or the care setting [4, 5].

In the last decades, the advances in medical technology fuelled the transition from front-to-front human treatment to technology-assisted training [6]. Rehabilitation technology has, indeed, witnessed an increasing succession of high-tech implementations in real-life settings, such as wearable devices, and robotic devices integrated with exergaming [7, 8], including Virtual Reality (VR) interfaces and Augmented Feedback modules [7, 9].

These latter developments are designed to integrate serious games into common rehabilitation tasks, empowering the therapist’s efficiency, patients’ motivation, adherence, and motor recovery [7, 10,11,12].

Besides, when it comes to practice, it is pivotal to ensure patients' comfort and well-being while approaching innovative yet unfamiliar devices [4, 8, 13].

Hence, research on rehabilitation technology is called to develop sound and validated methods to assess technology-assisted devices in terms of safety, end-user degrees of acceptance, adherence, and satisfaction [14,15,16,17]. Recent reviews reported how different and heterogeneous assessment tools were often used for the evaluation of Technology-Assisted Rehabilitation (TAR) [18, 19]. Furthermore, in previous works, patient-device interaction was assessed considering single and very specific issues, such as the patient’s emotional state [10, 11, 16, 17, 20], motivation [12, 21,22,23], psychosocial impact [24] and usability [25, 26] of the device adopted. In this regard, a limited number of instruments have already been applied [26,27,28,29], pertaining few distinct aspects of patient-device interaction [29]. As a result, according to our review, the current lack of a dedicated scale integrating the multiple features of TAR limits the comparison of findings regarding their effectiveness.

The aims of the present study are: (a) to develop a questionnaire on patients’ experience with rehabilitation and technology-assisted devices in a real-world clinical setting; (b) to administer the questionnaire to a pilot group of neurologic patients to assess its feasibility of administration and statistical properties.

Methods

Study design

The study was composed of two subsequent phases: questionnaire development through Delphi methodology and questionnaire application. In the second phase, an observational, feasible, multi-centric study relating to patients’ experiences with TAR was conducted in the Neurorehabilitation Unit of Istituti Clinici Scientifici Maugeri Spa SB- IRCCS Montescano, Italy, and at the Habilita Zingonia Centre of Ciserano, Italy.

Ethics approval and consent to participate

The study design and the protocol were submitted and approved by the Institutional Review Board and by the Ethics Committee (Comitato Etico Istituti Clinici Scientifici ICS Maugeri Pavia, approval CE number 2206 date 29.5.2018) and were implemented following the World Medical Association code of Ethics (Declaration of Helsinki, 1967).

Questionnaire development

A multidisciplinary team (neurologists, psychologists, bioengineers, physiatrists, and rehabilitation therapists specialized in technological devices for rehabilitation) was constituted to specifically work on the creation of the Technology Assisted Rehabilitation Patient Perception Questionnaire (TARPP-Q). The working board aimed to create a multidimensional, self-administrable, and close–ended questionnaire to evaluate the different facets of patient experience with TAR devices and exergaming.

The process followed a bottom-up approach started by freely observing inpatients during high-technology training; clinical observations prompted a literature revision on the development of questionnaires administered to assess patients’ experience with TAR. Based both on the existing literature and on clinical experience, the team independently and freely identified a pool of items that were then submitted to a Delphi methodology and progressively reviewed to reach a general agreement [30] (Fig. 1 Questionnaire construction process). The same procedure was applied to determine the questionnaire format, items’ number, questionnaire length, and rating scale degree.

Fig. 1
figure 1

Questionnaire construction process

The questionnaire was developed and administered in Italian and subsequently translated into English.

The full version of the questionnaire in English is provided for understandability purposes. Three independent investigators (AG, ST, RCa), whose native language was Italian, translated the original version of the questionnaire into English. An idiomatic translation was preferred over a word-for-word transposition. The investigators compared the 3 translated versions, with consensus. Lastly, an independent translator who had not participated in the first stage and whose native language was English supervised and finalized the last translation, with consensus.

Questionnaire application

The first version of the TARPP-Q, obtained through Delphi consensus, was administered over 6 months to consecutively admitted inpatients. Patients underwent rehabilitation training with TAR and exergaming at the third (T0) and the last training session (T1).

Sample

Inclusion criteria: in-patients, regardless of gender, aged between 18 and 85 years of age, admitted for a TAR program with exergaming due to a neurologic disease.

Exclusion criteria: cognitive deficits (MMSE ≤ 22) [31], insufficient knowledge of written or spoken Italian language, aphasia, severe visual or auditory deficits.

Data collection

The following data were collected at admission: age, years of education, type of disease, and disease duration. In addition, the FIM scale [32] was administered at T0 and T1. Patients were requested to give their informed consent to the study and the authorization of scientific treatment of their medical records in an anonymous form.

Technology-assisted training

According to their prevailing functional deficit, patients were subjected to a daily upper limb (effective 30-min) technology-assisted treatment or to a daily (30-min) technology-assisted gait training for 5 days/week for 4 weeks. The first and the second session were for adaptation. There was no break during gait session training, while there was a 1-min break between one exercise and another to select the following exercise by the operator during upper limb rehabilitation.

The training was conducted with exergames integrated with the upper-limb and gait devices. The exergames of the two devices used are defined by the vendor (Hocoma, Switzerland) as Augmented Performance Feedback (APF) activities. More specifically, the devices present an inbuilt software that can be used to enhance motivation by providing visual and interactive screen feedback on the ongoing motor performance. The active and live effort from the user is therefore represented on a display monitor (scores, accuracy rates, etc.) through different available games.

Three examples of exercises were below detailed (Figs. 2, 3, 4).

Fig. 2
figure 2

Armeo’s exercise example 1

Fig. 3
figure 3

Armeo’s exercise example 2

Fig. 4
figure 4

Lokomat’s exercise example

Upper limb rehabilitation

Upper limb-assisted training was performed by Armeo Spring ® (Hocoma, Switzerland), consisting of mechanical anti-gravity support equipped with 8 joints that permit movement in three-dimensional space; in addition, it is provided with a handle with which the patient can execute a grab gesture, thus allowing for the evaluation of the grip force. The ArmeoSpring ® provides real-life exergaming with APF, partially simulated activities of daily living, to conduct therapeutic exercises applicable in a safe environment. Namely, patients actively perform training in front of a screen, interacting with a semi-immersive environment.

Gait rehabilitation

Technology-assisted gait training was performed by Lokomat® (Hocoma, Switzerland) constituted by an exoskeleton, a body weight support, and a treadmill. It facilitates symmetric hip and knee gait patterns thanks to its exoskeleton, driven by programmable actuators.

Rehabilitative exercises with APF are carried out in a semi-immersive environment. During all training sessions, patients were placed in front of a screen where a representation of the patient simulated walking in a virtual landscape, guided by the patient’s movements.

Functional Independence Measure (FIM®)

The Functional Independence Measure (FIM®) scale measures the patient's disability level and indicates the degree of assistance required for the subject to carry out activities of daily living; it is largely adopted in rehabilitation as a functional outcome indicator [32]. It consists of 18 items, considering motor (13 items) and cognitive (5 items) functions, respectively. Each FIM item is scored on a 7-point Likert scale, where 1 indicates complete dependence and 7 represents complete independence. The total score ranges from 18 (complete dependence on all items) to 126 (complete independence assessed in all domains). Two sub-scores can be assessed: FIM motor (range 18–91) and FIM cognitive (range 0–35). All the evaluations were performed by certified health care professionals.

Statistical analysis

Descriptive statistics for individual TARPP-Q items and discrete variables are reported as numbers (percentage frequency).

To investigate the existence of an underlying structure in the interrelationships among the items of the questionnaire, exploratory factor analysis was carried out.

The principal factor method was used to extract factors, followed by orthogonal rotation (Varimax method) to assist in the interpretation of the factors and to ensure that they were uncorrelated.

The determination of the number of factors to extract was guided by theory (variance explained by each factor) and seeing which number of factors yielded theoretical meaningfulness and the most interpretable results. The determination as to what the underlying factor might represent was inferred from the variables significantly loaded on their factors. An absolute factor loading value ≥ 0.35 was considered to indicate that a variable contributed significantly to a factor.

The following were verified: at least three variables (items) had to load significantly on each retained factor, the variables that loaded on a given factor needed to share conceptual meaning, and the variables that loaded on different factors considered different constructs.

Internal consistency of the factors measured in the questionnaire was assessed using Cronbach’s alpha.

The association between TARPP-Q factor scores (values at admission, at discharge, and the difference between values at discharge and admission) and age, education, and FIM total and sub-score, were analyzed by correlation analysis (Spearman r).

As to the questionnaire application, within-group comparisons were carried out by the Wilcoxon signed-rank test.

All statistical tests were two-tailed and statistical significance was set at p < 0.01. All analyses were carried out using the SAS/STAT statistical package, release 9.4 (SAS Institute Inc., Cary, NC, U.S.A.).

Results

Questionnaire development

A final consensus was obtained on a 31-item TARPP-Q provisional questionnaire, constituted of 10 main questions, of which 5 with multiple answers. The TARPP-Q response to each item was rated using a four-level Likert-type scale (forced choice, no neutral option): strongly disagree (score 1), disagree (score 2), agree (score 3), strongly agree (score 4).

Questionnaire application

Forty-four patients were included in the study. Demographical and clinical data are reported in Table 1.

Table 1 Demographic and clinical variables

All patients answered the questionnaire autonomously. There were no unaccounted answers

The responses to individual TARPP-Q items at T0 and T1 are reported in Table 2. For the sake of completeness, the mean values ± SD of all the items are also reported.

Table 2 TARV-Q single item responses at T0 and T1

After the analyses, two items (“How would you describe your experience with the device to friends/relatives? Unsatisfied” and “Instructions given by the physiotherapist were useful”) were excluded from the ultimate version of the questionnaire. Specifically, the first item did not load to any of the four factors, then was removed due to its construct inconsistency. The second one was deleted as diverting the attention from the primary interest of the questionnaire—that is, specifically, the patient-device interaction—by introducing another variable, such as the operator’s role. The authors became aware of the inconsistency of this item within the construct coherence of the questionnaire during the patient’s assessment phase only, therefore with a unanimous agreement the item was removed from the questionnaire.

The final version of the questionnaire consisted of 10 questions, 5 with multiple answers, totaling 29 items.

At the factor analysis, four factors emerged, which were named as follows, after a further consensus amongst the members of the multidisciplinary team: Factor 1: Positive attitude, Factor 2: Usability, Factor 3: Hindrance perception, and Factor 4: Distress. The values of Cronbach’s alpha ranged from a moderate 0.72 (“Distress”, Factor 4) to a high 0.92 (“Positive attitude”, Factor 1). In Table 3 items’ analysis of the TARPP-Q is reported. Cronbach’s alpha variations, obtained by removing single items from the factor, are reported in rows for each item. When appropriate, reverse scoring was applied to selected items before their inclusion in the pertaining domain.

Table 3 Items and factors of TARV-Q

In Additional file 1 printable English version is available.

In Additional file 2 printable Original Italian version is available.

Descriptive statistics for the total TARPP-Q score and the FIM (motor, cognitive and total) at T0 and T1 and the 4 factors are reported in Table 4.

Table 4 Descriptive statistics for the 4 factors, the total TARPP-Q and FIM scores at T0 and T1

FIM scores and all TARPP-Q factors, except for Distress (p = 0.11), significantly improved at the end of the rehabilitation treatment.

The association between age, education and FIM scores, and TARPP-Q factors were also assessed, but no significant relationship was observed with any domain at T0 nor T1. Analogously, no significant association was observed between age, education, and changes (values at T1 – values a T0) in FIM scores vs changes in TARPP-Q factors.

Table 5 reports the correlation analysis between factors.

Table 5 Correlations between factors

Discussion

The assessment of high-technology devices according to patients’ perspectives is a relevant—and relatively young—topic for motor rehabilitation [2, 3]. Research studies proved that non-motor variables such as patient positive disposition, perception of safety, motivation, or engagement may have significant positive effects on the efficacy of technology-assisted training [2, 13, 16, 33]. However, it is currently under debate whether the use of high technology or robotic devices may also convey negative feelings, being felt by users as uncomfortable, disorienting, or even a source of fear [34].

The TARPP-Q was conceived to cover different facets of patients’ experiences identified through a Delphi methodology by a multidisciplinary team. Relevant determinants such as Positive attitude, Usability, Hindrance perception, and Distress were identified. Since most of the currently available high-technology devices are integrated with exergames [9], pertaining questions were also enclosed in the item set.

Questionnaire development

Overall, the TARPP-Q showed sound psychometric properties. Four factors were identified by an exploratory factor analysis, which resulted in coherence with clinical experience (construct validity) and proved to have a good internal consistency (sound values of total Cronbach’s alpha and of Cronbach’s alpha variations, obtained by removing single items from the factor).

Positive Attitude (Factor 1). The first factor had high internal consistency (Cronbach Coefficient Alpha = 0.92) and included items related to positivity, amusement, comfort, aid, and engagement. Given the nature of the item aggregation, the factor was labeled as “Positive attitude”—referring to patients’ positive set of emotions, beliefs, and behaviors toward the technology-assisted device. Although structural and ergonomic characteristics of devices are indeed drivers of treatment efficacy [35] and safety [14], patients’ Positive attitude could positively impact the device constant usage, preventing discontinuations and drop-offs. Literature findings suggested how personality traits and beliefs may also influence change-promotion behaviors while using devices [36, 37]; moreover, a sense of comfort (items 5A and 7A) is a prerequisite for a safe application of technology-assisted devices over time [14] and primary requirement for a stable motor recovery [38, 39]. In addition, amusement (items 5D and 7D) and enthusiasm (item 10A) also affect rehabilitation outcomes, particularly when related to the use of exergames [9, 40]. Similarly, engagement (items 8A, 9, and 10B) and well-being (item 6B) are known to impact the training, even if mediated by the type of training or device (repetition and variation) [41, 42] and by patient expectations [25]. Lastly, patient perception of aid (item 8C) while performing exercises is only partially covered in literature and deserved further investigation [43].

Usability (Factor 2). Usability can be defined as the capacity of a system to allow users to perform tasks safely, effectively, and efficiently while enjoying the experience [44]. Assessing usability in rehabilitation ensures device maximum functionality, whilst increasing effectiveness, engagement, and ease of learning. As a result, the usability of TAR devices has been widely investigated, with encouraging results as to patients and health care professionals’ device perception and execution of movements that are accurate, natural, and harmless for the patient [25, 44, 45]. Items about Usability emerged as a factor in TARPP-Q, showing a sound internal consistency (Cronbach Coefficient Alpha = 0.84) and collecting items investigating ease and enjoyableness of use. Interestingly, two items related to perceived performance improvement also loaded to Usability (4, 6E). On this note, is it worth noticing how a user-friendly device leads to positive perception (items 1, 2, 3), helping movement execution (items 4 and 6E) in a way to influence treatment motor outcomes and exergames scores [9, 46, 47].

Hindrance perception (Factor 3). The third factor found (Cronbach Coefficient Alpha = 0.73) collects both negative implications on motor execution of exercises (i.e. poor movement control; item 6D) and psychological facets that may interfere with device usage, such as feeling awkward (items 5E and 7E) and being uncomfortable (items 5B and 7B). In the current context, Hindrance perception is to be referred to as subjective sensations (constraint, impediment) deriving directly from “wearing” the exoskeleton while performing exercises, as already expressed in other works [48, 49]. It may be also inferred that “feeling” the device guiding movements and enveloping body parts might be perceived as something out of ordinary, determining a bizarre experience. Indeed, the present distinction deserves further clarification, especially regarding the subjective experience of motor control and body constraint. The emerged meaningful concept, however, witness the multi-level complexity of TAR [38], encouraging the adoption of a holistic perspective to treatment [25, 42]. Negative perceptions towards technological devices are indeed a wake-up call for health care professionals, as they might considerably interfere with motivation and result in treatment discontinuity or rejection [50].

Distress (Factor 4). Distress can be defined as a state of emotional suffering associated with stressors and demands that are difficult to cope with in daily life [51]. As a barrier to technology-assisted devices, Distress emerged as the fourth factor, showing a moderate internal consistency (Cronbach Coefficient Alpha = 0.72). Recent studies suggested conducting an in-depth analysis of the relationship between high-technology or robotic devices and psychological responses [52]. Consistently, assessing negative feelings and psychological domains could provide considerations of clinical interest (items 5F, 7F, 8B, 8D). In this regard, given the exploratory factor analysis, exergaming-items 8B (“Seeing the score reached on the screen: makes me feel under pressure”) and 8D (“Seeing the score reached on the screen: limits me”) (“Distress”) have been included to Factor 4, to further characterize the distinction emerged during the analyses: Factor 4 (Distress), mostly focusing on psychological shades of performance limitations and distress, versus awkwardness and physical sensation of impediment, pertaining Factor 3 (Hindrance perception).

Accordingly, items 8E (“makes me feel inadequate”), 5C, 7C (“clumsy”) and 6A (“discomfort”) evaluate psychological facets of a patient’s perception of TAR. However, further studies are needed to systematically investigate the role of specific psychological traits (such as self-esteem) and emotional statuses during the treatment by using tailored instruments [10, 33, 36, 42].

Finally, factor analysis showed that item 6C “fatigue” (“While exercising with the device, I experienced: fatigue”) loaded to Factor 4 (Distress). Fatigue is a central issue for motor rehabilitation, particularly in neurologic diseases such as Parkinson’s disease [53], multiple sclerosis [54], and stroke [55], and prominently affects treatment motivation and rehabilitation efficacy [56]. However, in the field of rehabilitation, fatigue, both in its definition and etiology, is still under debate. For this reason, the item “Fatigue”, which resulted included in Distress—and thus in a psychological factor—could be regarded as such for its psychological implications only. Future studies on fatigue together with large-scale applications of TARPP-Q may better clarify its role.

Questionnaire application

As to the questionnaire application, statistical analysis among the TARPP-Q factors, demographic, and clinical variables such as age, education, and patients’ disability level (expressed by FIM score at T0 and T1), identified no significant correlations. This result suggests that, regardless of age [15], socio-cultural context, and level of disability, patient technology acceptance is largely positive in our sample. In this respect, the age of the patient, for example, might be a conditioning factor in the acceptance of TAR [15]. In previous work, similar results were obtained by using the PIADS scale [24].

The four factors’ pre-post training changes (Table 5), obtained during the questionnaire application, further clarify this latter consideration: while FIM scores (Motor and Total subscales), Positive attitude, Usability, and Hindrance perception significantly improved at T1. Distress only remained unchanged after training.

The significant increase in Positive attitude and Usability factors is probably due to progressive experience determined by the training itself, whilst enhancing patient competence [41, 57]. As for the improvement of Positive Attitude, it is well known that patient perception of rehabilitation devices is associated with treatment satisfaction and not exclusively with physical improvement [10, 16]. Overall, results confirm the role of Positive attitude and Usability, while highlighting that patients’ psychological traits are crucial determinants in the interaction with a technology-assisted device [37]. Notably, in our sample, Usability resulted significantly correlated to patient Positive Attitude, proving the pivotal role of patient positive feelings towards the perceived usability of TAR.

Concerning both the factors Hindrance perception and Distress, the frequency distribution of TARPP-Q single item responses at T0 (Table 2) showed low levels of distress and negative perception. Consequently, fear and skepticism towards technology-assisted devices [34] resulted to be low or at least controlled in our sample. This might be due to different reasons, ranging from effective patient-healthcare professional communication to the personal characteristics of the participants. Future studies may furtherly investigate this theme, suggesting best practices to empower patients’ disposition during the interaction with TAR devices.

Interestingly, at present, factor analysis showed that items with a negative meaningful concept aggregate into two separate factors (Factor 3, Hindrance perception; Factor 4, Distress). Given the overall similarity of the contents, further studies are needed to better define the nature of the theoretical differences between these two factors.

As to patients’ perception of movement (6D. Poor control of my movements; 6E. Better control of my movements; 4. Movements (walking, use of arm) improved with the device), the TARPP-Q highlighted a betterment of both motor control and functionality. Indeed, clinical implications of movement control in TAR are noteworthy and still partially to be investigated [44, 45].

Finally, concerning the exergaming section of the questionnaire, factor analysis showed that pertaining items were distributed in separate factors (Factor 1, Factor 3, Factor 4). The non-aggregation may suggest that the role of exergaming in neurorehabilitation is multidimensional and still to be clarified, as a result of the complex combination of both motor and cognitive effects [58,59,60].

Limitations and future developments

TARPP-Q development was exclusively guided by collecting theoretical inputs from empirical observation and clinical experience. Nevertheless, the questionnaire aims to collect information from a relatively novel field [3, 18], an assumption that account for the exploratory nature of the study. The questionnaire, however, was administered in expert-lead settings, consisting of an interprofessional board of specialists in neurology, physiatry, physiotherapy, psychology, and bioengineering, whose strict and ongoing confrontation—together with the implementation of the Delphi procedure—may have strengthened its reliability.

It is important to highlight that the present results derive from the self-evaluation of patients who have carried out rehabilitation exclusively with fixed exoskeletons. Future studies may ascertain if similar considerations could be extended to other technologies (wearable, portable devices, etc.) which perform as motor “aids” to patients’ body parts during natural gait or upper-limb movements. Also, further studies are needed to determine the temporal stability of the TARPP-Q and to better define the nature of the two “negative” factors (Factor 3, Factor 4).

Given the multiple facets of exergaming and rehabilitation, future refinements of the TARPP-Q may consider extending the number of items dedicated to the theme (including a more structured reference to Virtual Reality and Augmented Reality), to provide a fine-grained analysis of the phenomenon [60, 61].

As to the pilot study, the main limitation is represented by the small sample, mainly composed of older people. However, it is well known that studies on this topic often consider a small number of patients. Indeed, a strength of the questionnaire relies on its multidisciplinary, as assessing general features of technology-assisted devices and exergames perception to describe both motor and emotional implications of patient-device interaction through a combination of determinants such as device usability, motor control, attitude, fatigue, and distress.

Lastly, the questionnaire was administered to patients who spoke Italian only and still needs full validation in terms of reliability, validity, and sensitivity. Rigorous validation of the TARPP-Q in the English language will be the object of a second study, possibly involving an international consortium so to allow comparison of validation results. Future developments of this questionnaire might also help shed light on the different nuances of meaning that might have been “lost in translation” during the Italian-English transposition.

Conclusions

The TARPP-Q aims to describe patients’ experience with Technology-Assisted Rehabilitation and exergaming. Results showed the role of four factors (Positive attitude, Usability, Hindrance perception, Distress) related to the direct patient experience with the device. Particularly, Usability is a pivotal parameter for patient performance in rehabilitation and it is directly correlated with patient Positive Attitude. Age, education, and disability level are not conditioning factors for patient experience with TAR.

Large-scale applications and full validation of the present questionnaire may clarify how patient perception affects training effectiveness, helping to customize device settings according to patients’ characteristics. It might be also suggestable to test the TARPP-Q in different clinical populations and across different TAR devices using exergaming.

Finally, the present contribution may hopefully help direct future investigations towards the often-unaccounted effect of patients’ psychological concerns [62], as the role of motor recovery and functional outcome largely prevail in the current research [39, 63]. Further investigations are needed to account for both those relevant facets—strongly bonded—and across different technology-assisted devices.

Availability of data and materials

Data and materials are available if requested.

Abbreviations

TARPP-Q:

Technology assisted rehabilitation patient perception questionnaire

TAR:

Technology-assisted rehabilitation

APF:

Augmented performance feedback

FIM:

Functional independence measure

MMSE:

Mini Mental State Examination

PIADS:

Psychosocial Impact of Assistive Device Scale

References

  1. Wade D. Adverse effects of rehabilitation—an opportunity to increase quality and effectiveness of rehabilitation. Clin Rehabil. 2009;23(5):387–93.

    Article  PubMed  Google Scholar 

  2. Boman IL, Bartfai A. The first step in using a robot in brain injury rehabilitation: patients’ and health care professionals’ perspective. Dis Rehabil Ass Tech. 2015;10(5):365–70.

    Article  Google Scholar 

  3. Beckerle P, Salvietti G, Unal R, Prattichizzo D, Rossi S, Castellini C, et al. A human-robot interaction perspective on assistive and rehabilitation robotics. Front Neurorobot. 2017;11:24. https://doi.org/10.3389/frbot.2017.00024.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Chen CY, Neufeld PS, Feely CA, Skinner CS. Factors influencing compliance with home exercise programs among patients with upper-extremity impairment. Am J Occup Ther. 1999;53(2):171–80.

    Article  CAS  PubMed  Google Scholar 

  5. Mayo NE. Patient compliance: practical implications for physical therapists. A review of the literature. Phys Ther. 1978;58(9):1083-1090.3 4.

    Article  CAS  PubMed  Google Scholar 

  6. Laut J, Porfiri M, Raghavan P. The present and future of robotic technology in rehabilitation. Curr Phys Med Rehabil Rep. 2016;4(4):312–9.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Beckerle P, Koiva R, Kirchner EA, Beckrater-Bodmann R, Dosen S, Christ O, et al. Feel-good robotics: requirements on touch for embodiment in assistive robotics. Front Neurorobot. 2018. https://doi.org/10.3389/frbot.2018.00084.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Krebs HI, Hogan N, Aisen ML, Volpe BT. Robot-aided neurorehabilitation. IEEE Trans Rehabil Eng. 1998;6(1):75–87.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Bergmann J, Krewer C, Bauer P, Koenig A, Riener R, Müller F. Virtual reality to augment robot-assisted gait training in non-ambulatory patients with a subacute stroke: a pilot randomized controlled trial. Eur J Phys Rehabil Med. 2018;54(3):397–407.

    Article  PubMed  Google Scholar 

  10. Bragoni M, Broccoli M, Iosa M, Morone G, De Angelis D, Venturiero V, et al. Influence of psychologic features on rehabilitation outcomes in patients with subacute stroke trained with robotic-aided walking therapy. Am J Phys Med Rehabil. 2013;92(10 Suppl 2):e16-25.

    Article  PubMed  Google Scholar 

  11. Calabrò RS, De Cola MC, Leo A, Reitano S, Balletta T, Trombetta G, et al. Robotic neurorehabilitation in patients with chronic stroke: psychological well-being beyond motor improvement. Int J Rehabil Res. 2015;38(3):219–25.

    Article  PubMed  Google Scholar 

  12. Colombo R, Pisano F, Mazzone A, Delconte C, Micera S, Carrozza MC, Dario P, Minuco G. Design strategies to improve patient motivation during robot-aided rehabilitation. J Neuroeng Rehabil. 2007;19(4):3.

    Article  Google Scholar 

  13. Kang CG, Chung MH, Chang MC, Kim W, Do KH. Views of physiatrists and physical therapists on the use of gait training robots for stroke patients. J Phys Ther Sci. 2016;28:202–6.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Feng Y, Wang H, Yan H, Wang X, Jin Z, Vladareanu L. J Research on safety and compliance of a new lower limb rehabilitation robot. Health Eng. 2017. https://doi.org/10.1155/2017/1523068.

    Article  Google Scholar 

  15. Eicher C, Haesner M, Spranger M, Kuzmicheva O, Graser A, Steinhagen-Thiessen E. Usability and acceptability by a younger and older user group regarding a mobile robot-supported gait rehabilitation system. Assist Technol. 2017. https://doi.org/10.1080/10400435.2017.1352051.

    Article  PubMed  Google Scholar 

  16. Stampacchia G, Rustici A, Bigazzi S, Gerini A, Tombini T, Mazzoleni S. Walking with a powered robotic exoskeleton: subjective experience, spasticity and pain in spinal cord injured persons. NeuroRehabilitation. 2016;39(2):277–83.

    Article  PubMed  Google Scholar 

  17. Phelan SK, Gibson BE, Wright FV. What is it like to walk with the help of a robot? Children’s perspectives on robotic gait training technology. Disabil Rehabil. 2015;37(24):2272–81.

    Article  PubMed  Google Scholar 

  18. Koumpouros Y. A Systematic review on existing measures for the subjective assessment of rehabilitation and assistive robot devices. J Healthc Eng. 2016. https://doi.org/10.1155/2016/1048964.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Monardo G, Pavese C, Giorgi I, Godi M, Colombo R. Evaluation of patient motivation and satisfaction during technology assisted rehabilitation: an experimental review. Games Health J. 2021;10(1):13–27.

    Article  PubMed  Google Scholar 

  20. Schiavolin S, Raggi A, Quintas R, Cerniauskaite M, Giovannetti AM, Covelli V, et al. Psychosocial difficulties in patients with Parkinson’s disease. Int J Rehabil Res. 2017;40(2):112–8.

    Article  PubMed  Google Scholar 

  21. Markland D, Hardy L. On the factorial and construct validity of the intrinsic motivation inventory: conceptual and operational concerns. Res Q Exerc Sport. 1997;68(1):20–32.

    Article  CAS  PubMed  Google Scholar 

  22. McAuley E, Duncan T, Tammen VV. Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: a confirmatory factor analysis. Res Q Exerc Sport. 1989;60(1):48–58.

    Article  CAS  PubMed  Google Scholar 

  23. Dishman RK, Ickes W. Self-motivation and adherence to therapeutic exercise. J Behav Med. 1981;4(4):421–38. https://doi.org/10.1007/BF00846151.

    Article  CAS  PubMed  Google Scholar 

  24. Fundarò C, Giardini A, Maestri R, Traversoni S, Bartolo M, Casale R. Motor and psychosocial impact of robot-assisted gait training in a real-world rehabilitation setting: a pilot study. PLoS ONE. 2018;13(2): e0191894. https://doi.org/10.1371/journal.pone.0191894. (eCollection 2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Swinnen E, Lefeber N, Willaert W, De Neef F, Bruyndonckx L, Spooren A, et al. Motivation, expectations, and usability of a driven git orthosis in stroke patients and their therapists. Top Stroke Rehabil. 2017;24(4):299–308. https://doi.org/10.1080/10749357.2016.1266750.

    Article  PubMed  Google Scholar 

  26. Muijzer-Wittveen H, Sibum N, Van Dijsseldonk R, Keijsers N, Van Assldonk E. Questionnaire results of user experiences with wearable exoskeletons and their preferences for sensory feedback. J Neuroeng and Rehab. 2018;15:112.

    Article  Google Scholar 

  27. Scherer MJ. Matching person and technology process and accompanying assessment instruments (revised edition). Webster, NY: The Institute for matching person and technology; 1998.

    Google Scholar 

  28. Demers L, Weiss-Lambrou R, Ska B. Item analysis of the Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST). Assist Technol. 2000;12(2):96–105. https://doi.org/10.1080/10400435.2000.10132015.

    Article  CAS  PubMed  Google Scholar 

  29. Kim T. Factors influencing usability of rehabilitation robotic devices for lower limbs. Sustainability. 2020;12(2):598. https://doi.org/10.3390/su12020598.

    Article  Google Scholar 

  30. Stewart J, O’Halloran C, Harrigan P. Identifying appropriate tasks for the preregistration year: modified Delphi technique. BMJ. 1999;319:224–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Folstein MF, Folstein SE, McHugh PR. Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98.

    Article  CAS  PubMed  Google Scholar 

  32. Heinemann AW, Linacre JM, Wright BD, Hamilton BB, Granger C. Relationships between impairment and physical disability as measured by the functional independence measure. Arch Phys Med Rehabil. 1993;74(6):566–73.

    Article  CAS  PubMed  Google Scholar 

  33. Zhong B, Niu W, Broadbent E, McDaid A, Lee TMC, Zhang M. Bringing psychological strategies to robot-assisted physiotherapy for enhanced treatment efficacy. Front Neurosci. 2019;18(13):984.

    Article  Google Scholar 

  34. Iosa M, Morone G, Cherubini A, Paolucci S. The three laws of neurorobotics: a review on what neurorehabilitation robots should do for patients and clinicians. J Med Biol Eng. 2016;36:1–11.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Wang X, Guo S, Qu H, Song M. Design of purely mechanical sensor–controller integrated system for walking assistance on an ankle foot exoskeleton. Sensors. 2019;19(14):3196.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Damholdt MF, Norskov M, Yamazaki R, Hakli R, Hansen Vesterager C, Vestergaard C, et al. Attitudinal change in elderly citizens toward social robots: the role of personality traits and beliefs about robot functionality. Front Psychol. 2015. https://doi.org/10.3389/fpsyg.2015.01701.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Schroeder AS, Von Kries R, Riedel C, Homburg M, Auffermann H, Blaschek A, et al. Patient-specific determinants of responsiveness to robot-enhanced treadmill therapy in children and adolescents with cerebral palsy. Dev Med Child Neurol. 2014;56(12):1172–9.

    Article  PubMed  Google Scholar 

  38. Mehrholz J, Pohl M. Electromechanical-assisted gait training after stroke: a systematic review comparing end-effector and exoskeleton devices. J Rehabil Med. 2012;44(3):193–9. https://doi.org/10.2340/16501977-0943.

    Article  PubMed  Google Scholar 

  39. Lo AC, Guarino PD, Richards LG, Haselkom JK, Wittenberg GF, Federman DG, et al. Robot-assisted therapy for long-term upper-limb impairment after stroke. N Engl J Med. 2010;362(19):1772–83. https://doi.org/10.1056/NEJMMoa0911341.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Tuena C, Pedroli E, Trimarchi PD, Gallucci A, Chiappini M, Goulene K, et al. Usability issues of clinical and research applications of virtual reality in older people: a systematic review. Front Hum Neurosci. 2020;14:93.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Hsieh YW, Wu CY, Lin KC, Yao G, Wu KY, Chang YJ. Dose–response relationship of robot-assisted stroke motor rehabilitation: the impact of initial motor status. Stroke. 2012. https://doi.org/10.1161/STROKEAHA.112.658807.

    Article  PubMed  Google Scholar 

  42. Manuli A, Maggio MG, Tripoli D, Gullì M, Cannavò A, La Rosa G, et al. Patients perspective and usability of innovation technology in a new rehabilitation pathway: an exploratory study in patients with multiple sclerosis. Mult Scler Related Disord. 2020;44: 102312. https://doi.org/10.1016/j.msord.2020.102312.

    Article  Google Scholar 

  43. Grüneberg P, Kadone H, Kuramoto N, Ueno T, Hada Y, Yamazaki M, et al. Robot-assisted voluntary initiation reduces control-related difficulties of initiating joint movement: a phenomenal questionnaire study on shaping and compensation of forward gait. PLoS ONE. 2018;13(3): e0194214. https://doi.org/10.1371/journal.pone.0194214.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Lee JY, Kim JY, You SJ, et al. Development and usability of a life-logging behavior monitoring application for obese patients. J Obes Metab Syndr. 2019;28(3):194–202. https://doi.org/10.7570/jomes.2019.28.3.194.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Tang S, Chen L, Barsotti M, Hu L, Li Y, Wu X, et al. Kinematic synergy of multi-DoF movement in upper limb and its application for rehabilitation exoskeleton motion planning. Front Neurorobot. 2019;29(13):99.

    Article  Google Scholar 

  46. Brutsch K, Schuler T, Koenig A, Zimmerli L, Koeneke SM, Lunenburger L, et al. Influence of virtual soccer game on walking performance in robotic assisted gait training for children. J Neuroeng Rehabil. 2010;22(7):15.

    Article  Google Scholar 

  47. Sakamaki I, Adams K, Medina MFG, Cruz JLC, Jafari N, Tavakoli M, et al. Preliminary testing by adults of a haptic-assisted robot platform designed for children with physical impairments to access play. Assist technol. 2018;30(5):242–50.

    Article  PubMed  Google Scholar 

  48. Poliero T, Lazzaroni M, Toxiri S, Di Natali C, Caldwell DG, Ortiz J. Applicability of an active back-support exoskeleton to carrying activities. Front Robot AI. 2020;7: 579963. https://doi.org/10.3389/frobt.2020.579963.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Näf MB, Junius K, Rossini M, Rodriguez-Guerrero C, Vanderborght B, Lefeber D. Misalignment compensation for full human-exoskeleton kinematic compatibility: state of the art and evaluation. ASME Appl Mech Rev. 2019;70(5): 050802. https://doi.org/10.1115/1.4042523.

    Article  Google Scholar 

  50. Corbianco S, Cavallini G, Dini M, Franzoni F, D’Avino C, Gerini A, et al. Energy cost and psychological impact of robotic-assisted gait training in people with spinal cord injury: effect of two different types of devices. Neurol Sci. 2021;42(8):3357–66. https://doi.org/10.1007/s10072-020-04954-w.

    Article  PubMed  Google Scholar 

  51. Arvidsdotter T, Marklund B, Kylén S, Taft C, Ekman I. Understanding persons with psychological distress in primary health care. Scand J Caring Sci. 2016;30(4):687–94. https://doi.org/10.1111/scs.12289.

    Article  PubMed  Google Scholar 

  52. Koenig A, Omlin X, Zimmerli L, Sapa M, Krewer C, Bolliger M, et al. Psychological state estimation from physiological recordings during robot-assisted gait rehabilitation. J Rehabil Res Dev. 2011;48(4):367–85. https://doi.org/10.1682/jrrd.2010.03.0044.

    Article  PubMed  Google Scholar 

  53. Chaudhuri A, Behan PO. Fatigue in neurological disorders. Lancet. 2004;363(9413):978–88.

    Article  PubMed  Google Scholar 

  54. Straudi S, Manfredini F, Lamberti N, Zamboni P, Bernardi F, Marchetti G, et al. The effectiveness of robot-assisted gait training versus conventional therapy on mobility in severely disabled progressIve MultiplE sclerosis patients (RAGTIME): study protocol for a randomized controlled trial. Trials. 2017;18(1):88. https://doi.org/10.1186/s13063-017-1838-2.

    Article  PubMed  PubMed Central  Google Scholar 

  55. De Groot MH, Phillips SJ, Eskes GA. Fatigue associated with stroke and other neurological conditions: implications for stroke rehabilitation. Arch Phys Med Rehabil. 2003;84(11):1714–20.

    PubMed  Google Scholar 

  56. Afshari M, Yang A, Bega D. Motivators and barriers to exercise in Parkinson’s disease. J Parkinsons Dis. 2017;7(49):703–11.

    Article  PubMed  Google Scholar 

  57. Metzger JC, Lambercy O, Califfi A, Dinacci D, Petrillo C, Rossi P, et al. Assessment-driven Selection and adaptation of exercise difficulty in robot-assisted therapy: a pilot study with a hand rehabilitation robot. J Neuroeng Rehabil. 2014;11(154):15.

    Google Scholar 

  58. Perez-Marcos D, Bieler-Aeschlimann M, Serino A. Virtual reality as a vehicle to empower motor-cognitive neurorehabilitation. Front Psychol. 2018;2(9):2120.

    Article  Google Scholar 

  59. Rogers JM, Dickworth J, Middleton S, Steenbergen B, Wilson PH. Elements virtual rehabilitation improves motor, cognitive, and functional outcomes in adult stroke: evidence from a randomized controlled pilot study. J Neuroeng Rehabil. 2019;16(1):56.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Tieri G, Morone G, Paolucci S, Iosa M. Virtual reality in cognitive and motor rehabilitation: facts, fiction and fallacies. Expert Rev Med Devices. 2018;15(2):107–17.

    Article  CAS  PubMed  Google Scholar 

  61. Moinuddin A, Goel A, Sethi Y. The role of augmented feedback on motor learning: a systematic review. Cureus. 2021;13(11): e19695. https://doi.org/10.7759/cureus.19695.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Novak D, Ziherl J, Olensek A, Milavec M, Podobnik J, Mihelj M, et al. Psychophysiological responses to robotic rehabilitation tasks in stroke. IEEE Trans Neural Syst Rehabil Eng. 2010;18(4):351–61.

    Article  PubMed  Google Scholar 

  63. Rodgers H, Bosomworth H, Krebs HI, van Wijck F, Howel D, Wilson N, et al. Robot assisted training for the upper limb after stroke (RATULS): a multicentre randomised controlled trial. Lancet. 2019. https://doi.org/10.1016/50140-6736819)31055-4.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The authors received no financial support for the research, authorship, and publication of this article.

Author information

Authors and Affiliations

Authors

Contributions

CFu, RCa conceived of the presented idea. CFu, RCa, SS, AG, ST, RCo developed the questionnaire; Rca, AG, ST translated the questionnaire; SS, MBa, CFe administered it to the pilot group. RM performed statistical analyses. CFu, RCa, ST, RM, AG, RCo, MBa, SS, MBu, CFe supervised the findings of this work. CFu, RCa, ST, RCu, AG, SS, MBa contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript. All authors discussed the results and contributed to the final version of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Cira Fundarò.

Ethics declarations

Ethics approval and consent to participate

The study design and the protocol were submitted and approved by the Institutional Review Board and by the Ethics Committee (Comitato Etico Istituti Clinici Scientifici ICS Maugeri Pavia, approval CE number 2206 date 29.5.2018) and were implemented following the World Medical Association code of Ethics (Declaration of Helsinki, 1967).

Consent for publication

All authors have participated in the work, have approved the final version of the manuscript, and agree with the content of the article. All authors consent for publication.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

TARPP-Q Questionnaire (English translation).

Additional file 2.

TARPP-Q Questionnaire (Original Italian version).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fundarò, C., Casale, R., Maestri, R. et al. Technology Assisted Rehabilitation Patient Perception Questionnaire (TARPP-Q): development and implementation of an instrument to evaluate patients’ perception during training. J NeuroEngineering Rehabil 20, 35 (2023). https://doi.org/10.1186/s12984-023-01146-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12984-023-01146-3

Keywords

  • Technology-assisted rehabilitation
  • Exergaming
  • Augmented performance feedback
  • Self-report
  • Patient experience
  • Questionnaire