- Open Access
Clinimetric properties of a novel feedback device for assessing gait parameters in stroke survivors
Journal of NeuroEngineering and Rehabilitation volume 11, Article number: 30 (2014)
Community-dwelling stroke survivors tend to become less physically active over time. There is no ‘gold standard’ to measure walking activity in this population. Assessment of walking activity generally involves subjective or observer-rated instruments. Objective measuring with an activity monitor, however, gives more insight into actual walking activity. Although several activity monitors have been used in stroke patients, none of these include feedback about the actual walking activity. FESTA (FEedback to Stimulate Activity) determines number of steps, number of walking bouts, covered distance and ambulatory activity profiles over time and also provides feedback about the walking activity to the user and the therapist.
To examine the criterion validity and test-retest-reliability of the FESTA as a measure of walking activity in patients with chronic stroke. To target the properties of the measurement device itself and thus exclude effects of behavioral variability as much as possible evaluation was performed in standardized activities.
Community-dwelling individuals with chronic stroke were tested twice with a test-retest interval varying from two days to two weeks. They performed a six-minute walk test and a standardized treadmill test at different speeds on both testing days. Walking activity was expressed in gait parameters: steps, mean-step-length and walking distance. Output data of the FESTA on the treadmill was compared with video analysis as the criterion measurement. Intraclass Correlations Coefficients (ICCs) and Mean Relative Root Squared Error (MRRSE) were calculated.
Thirty-three patients were tested to determine criterion validity, 27 patients of this group were tested twice for test-retest reliability. ICC values for validity and reliability were high, ranging from .841 to .972.
This study demonstrated good criterion validity and test-retest-reliability of FESTA for measuring specific gait parameters in chronic stroke patients. FESTA is a valid and reliable tool for capturing walking activity measurements in stroke, and has applicability to both clinical practice and research.
In many Western nations, stroke is a leading cause of death and serious long-term disability . A frequent consequence of stroke is unilateral loss or limitation of muscle function, leading to a loss of mobility, movement and functional ability [2, 3]. Van de Port et al. (2006) showed that a substantial proportion of community-dwelling stroke survivors becomes less physically active over time . Post-stroke physical inactivity may produce physical deconditioning, and as a consequence a decline in function . A decline in function reduces participation in the community and quality of life  and decreases independence of the stroke survivor . Furthermore, physical inactivity increases the risk of developing co-morbidities and having a recurrent stroke . Accurate measurement of real life walking activity could be beneficial in tailoring rehabilitation. Using actual performance data and providing feedback might support self-management strategies to prevent physical and functional decline and subsequent consequences.
Currently assessment of walking activity generally involves subjective or observer-rated instruments . These instruments have disadvantages such as the risk of recall bias, social desirability of answers, and poor generalisation . Objective assessment of the number of steps can be done with pedometers. Roos et.al. (2012) demonstrated the disadvantage of measuring only the total number of steps taken . They found differences in walking bouts and walking time between older adults and stroke survivors and that it varied based on functional ability. This relevant variation could not have been identified when measuring only steps per day . Measuring gait parameters with accelerometers overcomes the limitation of measuring only the number of steps. To measure gait parameters by accelerometry in this population specific algorithms are required since stroke survivors are slow walkers  and accuracy of detecting steps decreases when gait speed and step frequency decrease . To date, several motion sensors have been used [11, 12], such as the accelerometer based StepWatch Activity Monitor (SAM) which has good validity in measuring gait parameters in stroke survivors. However, current devices are not capable of providing feedback to the stroke survivor about their walking activity. Providing feedback about their walking activity might prevent physical inactivity, and as a consequence a decline in function . To monitor walking and to investigate potential beneficial effects of feedback in stroke survivors we developed FESTA.
FESTA (FEedback to STimulate Activity) is a telemetric system that includes a tri-axial piezo capacitive accelerometer which can be coupled to a docking station. The docking station is capable of; calculating gait parameters, evaluating whether the amount of walking activity during the day was sufficient according to the goal set by the physical therapist, providing the feedback to a screen visible for the stroke patient, sending an email towards the physical therapist with the calculated gait parameters and recharging the battery of the accelerometer to continue monitoring the next day.
As measuring gait parameters is more challenging in stroke survivors, the first step in this developing process was to examine the criterion validity and test-retest reliability of FESTA at gait parameter recognition in chronic stroke survivors using a stroke specific developed algorithm. We examined gait parameters; number of steps, mean-step-length with a standardized treadmill test and walking distance with an over ground 6 minute walk test. Furthermore FESTA calculates walking time and walking bouts as a derivative from steps .
A convenience sample of community-dwelling, chronic stroke survivors was recruited from ten private physical therapy practices, the daycare center of ‘Zorgspectrum’ and the patients’ association ‘Samen verder’ in the Netherlands and the University of Maryland in the United States of America. Stroke was defined according to the World Health Organization definition. Participants were able to walk independently without physical assistance (Functional Ambulation Categories score ≥ 3)  and were at least three months post stroke.Participants were excluded if they had severe cognitive disorders (Mini-Mental State Examination <24) , severe communicative disorders (Utrechts Communicatie Onderzoek <4)  or acute disorders impairing gait. All participants gave written informed consent prior to participation in the study. The research protocol and all informational material were approved by the Medical Ethical Committee (MEC) of the University Medical Center Utrecht and the Institutional Review Board of the University of Maryland, Baltimore. Treatment of the participants was according to the Helsinki declaration .
Equipment & experimental protocol
Participants were tested twice with a test-retest interval of a minimum of two days and a maximum of two weeks using the six-minute walk test (6MWT) and a standardized treadmill test. At baseline, inclusion measurements and collection of personal and anthropometric data were performed prior to the physical tests.
During both tests the FESTA was worn around the back site of the waist, between the posterior superior iliac spines. The FESTA contains one tri-axial, piezo-capacitive accelerometer (70*80*25 mm, 150 grams, range ± 2.5 g). Based on sensor alignment, acceleration signals were identified as anterior-posterior (AP), medio-lateral (ML) and vertical (VT). Output is in mV, a change of 1 mV corresponded to a change of 0.08 m/s2 (resolution)). Acceleration signals were digitally stored on a memory card with a sampling rate of 25 samples/s.
The 6MWT was performed to assess overground walking distance. The 6MWT was performed according to the American Thoracic Society Guidelines . Walked distance was determined by counting the number of walked laps (20 meters) and adding any final fraction of laps, measured by a measuring wheel. Results were used to calculate the comfortable walking speeds for the treadmill test (CWT) and to assess the overground walking distance validity and reliability of the FESTA.
Standardized treadmill test
Gait parameters, number of steps and mean-step-length were determined using a standardized treadmill test. Because accuracy of the gait parameter; steps recognition depends on gait speed and gait speed may vary during a day and is low in this population  we executed a treadmill test at three different gait speeds within each subject. Gait speeds were established at 15% below, equally to and 15% above comfortable gait speed. Each speed condition lasted for two minutes. The mean walking speed measured by the 6MWT −10% was used to define the comfortable walking speed. Fingertip handrail support was allowed during testing. The treadmills (En Mill treadmill, Enraf Nonius, the Netherlands and Gait Trainer 3™, Biodex, USA) were calibrated prior to the study. A camera was placed 1.2 meter behind the treadmill (Panasonic type HC-V70, 50 samples/s).
Data processing and algorithms
From every block of two minutes at different speeds, only the last 90 seconds were analysed. The researcher counted the number of steps during these blocks of 90 seconds from the video afterwards and was blinded from the results of FESTA. Distances from the treadmill test were determined by using the treadmill speed and the testing time of the treadmill test. The mean-step-length was derived from the distance walked and divided by the steps taken by both legs.
From the same blocks of 90 seconds, the gait parameters (number of steps and mean-step-length) from FESTA were analysed using Matlab (Matlab 7.10.1, The MathWorks Inc, USA).For the step detection we used spectral analysis derived from the AP acceleration signal. Taking the individual variety of the distance-acceleration relationship into account, we used an individual calibration procedure for distance measures to determine the acceleration-distance relation . Firstly we calculated the root mean square of the AP acceleration signal, secondly conducted a linear fit (first order polynomial) between the different gait speeds and the different root mean square values, thirdly we used the polynomial function to predict the walking speed and subsequently walking distance in the treadmill test and 6MWT.This distance prediction derived from a single acceleration signal and the individual calibration procedure is described by Schutz et al. (2002)  in more detail.
To assess the validity and reliability of FESTA, we compared the gait parameters derived from FESTA with the gold standard (video analysis). Comparisons for the gait parameters number of steps and mean-step-length were performed by using the data from the treadmill test. This procedure is consistent with procedures from similar validation studies [20–23] and video analysis seems to be the most appropriate criterion standard for the assessment of physical activity . The comparison of the gait parameter walking distance was performed using the data derived from the 6MWT.
Descriptive statistics were performed for all variables and normality was assessed by visual inspection of histograms and quantile-quantile plots. An ICC3,1 of ≥ .75 was defined as high as suggested by Burdock et al.(1963) . All calculations were performed using SPSS (IBM Software, SPSS Statistics 20, USA) or Matlab (Matlab 7.10.1, The MathWorks Inc, USA).
To assess the level of agreement between FESTA and the gold standard, and thus the criterion validity, single measures intraclass correlation coefficientsagreement (ICC3,1, Two-way mixed model) were calculated for the different gait parameters; number of steps and mean-step-length obtained from the treadmill test and over ground walked distance in the 6MWT. Furthermore the Mean Relative Root Square Error (MRRSE) was calculated for each parameter. The MRRSE is a measure of the differences between the values of FESTA and the observed values, relative to the unit of measurement (see Formula). The MRRSE gives an indication of the mean error of FESTA per step or number of steps as a percentage of the measurement unit.
= mean of the observed values, criterion measurement, video analysis
Xmo del,i= values obtained by FESTA
Single measures intraclass correlation coefficientsconsistency (ICC3,1, Two-way mixed model) was calculated to analyse the test-retest reliability of FESTA. Additionally, the Minimal Detectable Change (MDC95) was calculated from the Standard Error of Measurement (SEM) as MDC95 = [1.96*SEMconsistency *√2] and SEM = [sd * , where r is the test-retest reliability coefficient ICCagreement 3,1 and sd is the standard deviation of the scores at the first test occasion (T0). The SEM is multiplied by 1.96 to determine the 95% confidence interval and multiplied by the square root of 2 to account for the additional error associated with repeated measurements . The MDC95 is the minimal amount of change that must be observed before the change can be considered to exceed the variation and measurement error at the 95% confidence level.
A total of 33 participants (17 men and 16 women) were tested and their data were used to determine the criterion validity of FESTA. Twenty-seven participants were tested twice. The other six participants did not perform a second test, due to motivational problems to perform a second test or being unable to perform a second test within the set time limit of two weeks after the first test. The mean age of the 33 participants was 61.8 ± 8.8 years, time since stroke was 5.6 years ± 3.8 years and the functional ambulation category (FAC) scores ranged from 3 to 5 (mean 4.4 ± 0.7). The average distance walked in the 6MWT was 317.3 meters, which is 0.88 m/s, ranging from 36 to 580 meters. For the treadmill testing, the different walking speeds varied from 0.08 to 1.5 m/s.
For number of steps and mean-step-length at the three different gait speeds, ICCagreement3,1 varied between 0.841 and 0.971 (p ≤ 0.001 for all values). Mean Relative Root Squared Errors (MRRSE) ranged between 3.4 and 9.1%. All agreement parameters are presented in Table 1.
Figure 1 illustrates the differences between the gold standard and FESTA for the gait parameters steps and mean-step-length, with the difference in steps (top panel) and mean-step-length (bottom panel).
Criterion validity for overground walking distance during the 6MWT is presented in Table 2. The difference between measured and estimated over ground walking distance in meters averaged −20.1 meters see Figure 2.
Table 3 presents the test-retest reliability for the gait parameters steps and mean-step-length, including ICC values and MDC. ICCconsistency 3,1 scores ranged from 0.876 to 0.972 and were all significant at p ≤ 0.001.
Figure 3 illustrates the differences between the first and second test occasion, reliability of the gait parameters; steps (top panel) and mean-step-length (bottom panel).
Test-retest reliability for over ground distance covered during the 6MWT for ICCagreement 3,1 ,was .97. Mean difference in meters was 8.1 meter, see Figure 4.
The objective of this study was to examine the criterion validity and test-retest reliability of the novel telemetric system, FESTA, in measuring walking activity in stroke survivors. To this end, we tested gait parameters; steps, mean-step-length and walking distance in chronic stroke survivors. Results of criterion validity and test-retest reliability indicate good validity and reliability as all ICC values were between .841 and .972. These results are similar to the most commonly used accelerometer in the stroke population [10, 12, 26]. Moreover, the results present higher accuracy in comparison to algorithms not specifically developed for the stroke population .
No clear trend can be seen between ICC values and MRRSE and the three gait speed conditions. This indicates that the validity of the FESTA is not affected by gait speed. Although the latter finding demonstrates the possible robustness of the FESTA for real-life use, we have to take into account that gait parameters differ for treadmill walking and over ground walking . When walking on a treadmill, the gait patterns of chronic stroke survivors are more symmetrical and stable compared to overground walking. Furthermore in real-life gait speed may vary during a day and even within a walking bout. Therefore the gait parameters steps and mean-step-length have to be interpreted with caution since these parameters were only tested on the treadmill and might not be generalizable to walking overground. Further research is needed to determine these outcomes in overground walking. Another limitation of the study was the test-retest reliability design. Although all conditions were similar in the first and second test occasion and subjects were stable, subjects did perform slightly differently in the first and second test. For example, subjects took slightly fewer steps in the treadmill test and walked a few meters further in the second 6MWT compared to the first test occasion. This affected the reliability results of FESTA.
For specific measurement devices, measurement errors should be smaller than the Minimal Clinically Important Difference (MCID) to detect a valuable clinical effect for individuals. For the treadmill tests, no MCIDs have been defined. The MDC95 score of the FESTA for overground walked distance at the 6MWT was 62.2 meters. Unfortunately the interpretation of this value remains unclear as the MDC of FESTA depends on de MDC of the 6MWT since the MDC of FESTA is derived during the 6MWT. The MDC of the 6MWT for this population ranges from 29 m  towards 54.1 m .
Previous studies with a similar design and aim as we had [11, 12, 30, 31] expressed accuracy performance in ICC values and Limits of Agreement. In this study, we added a new measure for validity; the Mean Relative Root Squared Error (MRRSE). It is known that ICC values are strongly influenced by the magnitude of the variance within the study sample. Furthermore, other than the name ICCagreement suggests, the ratio of variances is calculated, rather than the absolute agreement score . When taking a closer look at the ICC formula, it is clear that a large variance in subject scores, as is the case in this study, will lead to a higher ICC . Studies with different variances in their study populations can therefore not be compared directly. To get a better insight in the true agreement between the output of the FESTA and the ‘gold standard’, and to eliminate the effect of the high variance in our study population, we calculated the MRRSE for each gait parameter. The MRRSE represents the mean absolute percentage difference between the two measurement devices, expressed in the percentage of unit of the parameter. This score is easy to use in daily practice, easy to interpret and not dependent of variance between patients. Therefore, we hereby suggest using the MRRSE in future research, as it provides a more direct comparison between studies and between measurement devices.
FESTA (FEedback to STimulate Activity) is a newly developed telemetric system and validity and reliability were shown to be good in the present analysis. It is designed to monitor and stimulate stroke survivors with respect to their daily walking activity. The physical therapist is able to interact by setting walking activity goals based on walking time and walking bouts. FESTA has several advantages over other methods for assessing walking activity; it can measure different gait parameters such as number of steps, mean-step-length, distance and as derivatives walking time and the number of walking bouts , whereas a step-counter can only determine the number of steps. Roos et. al. (2012) clearly stated that steps alone is not sufficient to characterize physical inactivity in stroke survivors . Due to the docking station FESTA is not limited by battery life and data capacity. Therefore it is able to monitor for a long time period without recharging or removing data. Furthermore FESTA provides the researcher, physical therapist and stroke survivor with real-life walking activity information. Future research will involve studying the effect of giving feedback using this device. The aim will be to increase walking activity by providing feedback to the user and providing information of actual walking activity and the daily pattern of walking activity to the physical therapist. Using FESTA provides new possibilities to measure walking activity of chronic stroke survivors in a valid and reliable way and thereby offers a variety of perspectives for research and treatment in this population.
Based on ICC values and MRRSE, this study demonstrated good criterion validity and test-retest reliability of the telemetric system FESTA for measuring gait parameters in chronic stroke survivors. FESTA provides the possibility to measure gait parameters in a valid and reliable manner and can be used, in both clinical practice and academic research.
FEedback to STimulate Activity
Intra class correlation
Standard error of measurement
Minimal detectable change
Mean relative root squared error
Six minute walk test
Comfortable walking speed for treadmill testing.
English C, Hillier S: Circuit class therapy for improving mobility after stroke: a systematic review. J Rehabil Med Off J UEMS Eur Board Phys Rehabil Med 2011,43(Suppl 7):565-571.
Indredavik B, Rohweder G, Naalsund E, Lydersen S: Medical complications in a comprehensive stroke unit and an early supported discharge service. Stroke 2008,39(Suppl 2):414-420.
Langhorne P, Stott DJ, Robertson L, MacDonald J, Jones L, McAlpine C, Dick F, Taylor GS, Murray G: Medical complications after stroke: a multicenter study. Stroke 2000,31(Suppl 6):1223-1229.
van de Port IGL, Kwakkel G, van Wijk I, Lindeman E: Susceptibility to deterioration of mobility long-term after stroke: a prospective cohort study. Stroke 2006,37(Suppl 1):167-171.
Brazzelli M, Saunders DH, Greig CA, Mead GE: Physical fitness training for stroke patients. Cochrane Database Syst Rev(online) 2011, 11: 1-109.
Pound P, Gompertz P, Ebrahim S: A patient-centred study of the consequences of stroke. Clin Rehabil 1998,12(Suppl 4):338-347.
Pearson OR, Busse ME, van Deursen RWM, Wiles CM: Quantification of walking mobility in neurological disorders. QJM 2004,97(Suppl 8):463-475.
Roos MA, Rudolph KS, Reisman DS: The structure of walking activity in people after stroke compared with older adults without disability: a cross-sectional study. Phys Ther 2012,92(Suppl 9):1141-1147.
Goldie PA, Matyas TA, Evans OM: Deficit and change in gait velocity during rehabilitation after stroke. Arch Phys Med Rehabil 1996, 77: 1074-1082. 10.1016/S0003-9993(96)90072-6
Taraldsen K, Askim T, Sletvold O, Einarsen EK, Bjåstad KG, Indredavik B, Helbostad JL: Evaluation of a body-worn sensor system to measure physical activity in older people with impaired function. Phys Ther 2011,91(Suppl 2):277-285.
Saremi K, Marehbian J, Yan X, Regnaux JP, Elashoff R, Bussel B, Dobkin BH: Reliability and validity of bilateral thigh and foot accelerometry measures of walking in healthy and hemiparetic subjects. Neurorehabil Neural Repair 2006,20(Suppl 2):297-305.
Mudge S, Stott NS, Walt SE: Criterion validity of the StepWatch Activity Monitor as a measure of walking activity in patients after stroke. Arch Phys Med Rehabil 2007,88(Suppl 12):1710-1715.
Orendurff MS, Schoen JA, Bernatz GC, Segal AD, Klute GK: How humans walk: bout duration, steps per bout, and rest duration. J Rehabil Res Dev 2008,45(Suppl 7):1077-1090.
Holden MK, Magliozzi MR, Nathan J, Piehl-Baker L: Clinical gait assessment in the neurologically impaired: reliability and meaningfulness. Phys Ther 1984,64(Suppl 1):35-40.
Folstein FSE, McHugh MF: “Mini-mental state". a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975,12(Suppl 3):189.
Back PMWMFZ, van der Velden H, Schepers VPM, Visser-Meily JMA: De Spontane Communicatieschaal van het Utrechts Communicatie Onderzoek: Een valide screener van communicatieve vaardigheden. Revalidata 2006,28(Suppl 133):p5.
World Medical Association, 2002 Declaration of Helsinki: Ethical principles for medical research involving human subjects. J Postgrad Med 2002,48(suppl 3):206-208.
ATS Committee on Proficiency Standards for Clinical Pulmonary Function Laboratories: ATS statement: guidelines for the six-minute walk test. Am J Respir Crit Care Med 2002,166(Suppl1):111-117.
Schutz Y, Weinsier S, Terrier P, Durrer D: A new accelerometric method to assess the daily walking practice. Int J Obes 2002, 26: 111-118. 10.1038/sj.ijo.0801856
Terrier P, Aminian K, Schutz Y: Can accelerometry accurately predict the energy cost of uphill/downhill walking? Ergonomics 2001,44(suppl 1):48-62.
Houdijk H, Appelman FM, Van Velzen JM, Van der Woude LH, Van Bennekom CA: Validity of DynaPort GaitMonitor for assessment of spatiotemporal parameters in amputee gait. J Rehabil Res Dev 2008,45(suppl 9):1335-1342.
Bussmann JB, Martens WL, Tulen JH, Schasfoort FC, van den Berg-Emons HJ, Stam HJ: Measuring daily behavior using ambulatory accelerometry: the activity monitor. Behav Res Methods Instrum Comput 2001,33(suppl 3):349-56.
Sirard JR, Pate RR: Physical activity assessment in children and adolescents. Sports Med 2001,31(suppl 6):439-454.
Burdock EI, Fleiss JL, Hardesty AS: A new view of inter-observer agreement. Pers Psychol 1963,16(suppl 4):373-384.
Haley SM, Fragala-Pinkham MA: Interpreting change scores of tests and measures used in physical therapy. Phys Ther 2006,86(suppl 5):735-743.
Fulk GD, Combs SA, Danks KA, Nirider CD, Raja B, Reisman DS: Accuracy of two activity monitors in detecting steps in people with stroke and traumatic brain injury. Phys Ther 2014,94(suppl xxx):1-34.
Harris-Love ML, Forrester LW, Macko RF, Silver KH, Smith GV: Hemiparetic gait parameters in overground versus treadmill walking. Neurorehabil Neural Repair 2001,15(suppl 2):105-112.
Fulk GD, Echternach JL, Nof L, O’Sullivan S: Clinometric properties of the six-minute walk test in individuals undergoing rehabilitation poststroke. Physiother Theory Pract 2008,24(suppl 3):195-204.
Liu J, Drutz C, Kumar R, McVicar L, Weinberger R, Brooks D, Salbach NM: Use of the six-minute walk test poststroke: is there a practice effect? Arch Phys Med Rehabil 2008,89(Suppl 9):1686-1692.
Edbrooke L, Lythgo N, Goldsworthy U, Denehy L: Can an accelerometer-based monitor be used to accurately assess physical activity in a population of survivors of critical illness? Global Journal of Health Science 2012,4(suppl 3):98-107.
Hale LA, Pal J, Becker I: Measuring free-living physical activity in adults with and without neurologic dysfunction with a triaxial accelerometer. Arch Phys Med Rehabil 2008,89(suppl 9):1765-1771.
McGraw KO, Wong SP: Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996,1(suppl 1):30-46.
We would like to thank the research assistants Japie Bakers, Jolanda Spruit on behalf of Hogeschool Utrecht and Nathan Turner and Alyssa Stookey on behalf of University of Maryland for their contribution in data collection. We also would like to thank our financial funder SIA RAAK, 2010-2-024 INT.
The authors state that there was no conflict of interests with any financial or personal relationships or organizations that could influence the research results.
MP has made substantial contribution in the experimental design, algorithm development, interpretation and analysis of data and drafting the manuscript. BA has made substantial contribution in analyzing the data and drafting the manuscript. IP has made substantial contribution in facilitating the research project and revising the manuscript. JD has made substantial contribution in revising the manuscript. KM has made substantial contribution in acquisition in subjects and revising the manuscript. JO has made substantial contribution in acquisition in subjects and revising the manuscript. HW has made substantial contribution in facilitating the research project, drafting and revising the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Punt, M., van Alphen, B., van de Port, I.G. et al. Clinimetric properties of a novel feedback device for assessing gait parameters in stroke survivors. J NeuroEngineering Rehabil 11, 30 (2014). https://doi.org/10.1186/1743-0003-11-30
- Step activity