Variables influencing wearable sensor outcome estimates in individuals with stroke and incomplete spinal cord injury: a pilot investigation validating two research grade sensors
Journal of NeuroEngineering and Rehabilitation volume 15, Article number: 19 (2018)
Monitoring physical activity and leveraging wearable sensor technologies to facilitate active living in individuals with neurological impairment has been shown to yield benefits in terms of health and quality of living. In this context, accurate measurement of physical activity estimates from these sensors are vital. However, wearable sensor manufacturers generally only provide standard proprietary algorithms based off of healthy individuals to estimate physical activity metrics which may lead to inaccurate estimates in population with neurological impairment like stroke and incomplete spinal cord injury (iSCI). The main objective of this cross-sectional investigation was to evaluate the validity of physical activity estimates provided by standard proprietary algorithms for individuals with stroke and iSCI. Two research grade wearable sensors used in clinical settings were chosen and the outcome metrics estimated using standard proprietary algorithms were validated against designated golden standard measures (Cosmed K4B2 for energy expenditure and metabolic equivalent and manual tallying for step counts). The influence of sensor location, sensor type and activity characteristics were also studied.
28 participants (Healthy (n = 10); incomplete SCI (n = 8); stroke (n = 10)) performed a spectrum of activities in a laboratory setting using two wearable sensors (ActiGraph and Metria-IH1) at different body locations. Manufacturer provided standard proprietary algorithms estimated the step count, energy expenditure (EE) and metabolic equivalent (MET). These estimates were compared with the estimates from gold standard measures. For verifying validity, a series of Kruskal Wallis ANOVA tests (Games-Howell multiple comparison for post-hoc analyses) were conducted to compare the mean rank and absolute agreement of outcome metrics estimated by each of the devices in comparison with the designated gold standard measurements.
The sensor type, sensor location, activity characteristics and the population specific condition influences the validity of estimation of physical activity metrics using standard proprietary algorithms.
Implementing population specific customized algorithms accounting for the influences of sensor location, type and activity characteristics for estimating physical activity metrics in individuals with stroke and iSCI could be beneficial.
Ubiquitous estimation of physical activity and quality of life measures collected from real life environments are becoming an imperative component to monitor successful translation of clinical research into patient’s own living environment [1,2,3]. With context to this, continuous monitoring paradigms are gaining substantial significance in tracking a patient’s compliance to a stipulated exercise regime and to gauge their level of community integration post rehabilitation [1, 2, 4]. Commonly used methods of measuring mobility include the traditional methods of performance-based or patient-reported measures. These measurements are either limited by rater or recall bias, or in their ability to encompass all aspects of community mobility. Advanced methods like camera based motion capture, pressure sensor walkway  and force plate systems for assessments, although significantly reliable, limits data collection to a confined controlled laboratory space and are expensive . While such controlled environment tests can provide high-resolution information to uncover the underlying biomechanics during in-patient movement assessments, they provide very little to no information about a patient's natural physical activity behavior and compliance in their community or home setting .
Evidence suggests that employing wearable sensors provide means to remotely and continuously track patient’s recovery in real world settings [7, 8]. Indeed, activity monitoring with wearables are showing cost benefits for healthcare and also paving the way for participatory clinical decision making for customized healthcare [1, 2, 9,10,11]. Despite offering numerous benefits, the validity and reliability of outcome estimates from these wearable sensors for rehabilitation medicine in individuals with chronic conditions is a daunting challenge for researchers [12, 13].
A potential reason for this being that most of standard proprietary algorithm (SPA) provided by sensor manufacturer’s are derived using empirical data from healthy populations leading to inaccurate/unreliable estimates when deployed to estimate outcome measures in clinical populations . Research to date acknowledges two major contributing factors to such estimation inaccuracies , namely, (i) the sensor location and (ii) variation in acceleration thresholds due to pathology specific movement signatures in comparison to healthy controls.
In this context, there has been less focus on studying the influence of such factors on outcome metrics other than step count, like, energy expenditure (EE) and metabolic equivalents (MET), especially in individuals with stroke and incomplete spinal cord injury (iSCI) [13, 15, 16]. Further, there is limited information regarding the relationship between attributes like (i) sensor type (i.e. body worn, body stuck and standalone Vs fusion sensor modalities) and (ii) characteristics of activity being studied, on the validity of outcome measures estimated using SPAs in populations with neurological impairments like iSCI and stroke.
Consequently, our primary goal was to investigate the influence of sensor type, and sensor location on the validity of physical activity outcome estimates (step count, energy expenditure (EE) and metabolic equivalent (MET) as provided by SPA from wearable sensors in a sample of healthy individuals (controls) and individuals with iSCI (ambulatory) and stroke. A secondary goal was to investigate the influence of activity characteristics (intensity) on the validity of each of the physical activity outcome estimates in our sample. In this pilot study, we investigated the influence of above aspects on validity of the outcome measures as provided by the respective SPAs from two research grade sensors. The spectrum of activities studied were identical to those encountered in activities of daily living (ADL), but were performed in a controlled laboratory setting. Although performed in a controlled environment, such findings will have important implications for understanding the possible factors that needs to be considered while estimating outcome measures using wearables in free living conditions.
It was postulated that the choice of sensor type, sensor location (arm, waist and ankle), population specific movement signatures (healthy, iSCI (ambulatory), stroke) and the characteristics of the activity being studied will significantly influence the validity of the physical activity outcomes metrics estimated using SPA in laboratory conditions.
The participant pool included, healthy controls, and individuals with iSCI and chronic stroke who could ambulate with or without an assistive device. Detailed group wise demographic information is provided in Table 1. Exclusion criteria included, (i) presence of any known serious cardiac conditions, (ii) neurological degenerative pathologies as co-morbidities (such as Multiple Sclerosis, Alzheimer’s disease, Parkinson’s disease, etc.), and (iii) inability to sit unsupported. In addition, subjects were requested to stay off of any medications that has previously known to affect their metabolism during the study period.
Currently, a plethora of commercial and research grade wearable devices are available for clinicians to choose from . Testing the validity of outcome metrics from all the devices was outside the scope of this study design. Therefore, in order to test the postulated hypotheses two specific research grade wearable sensor types used in clinical research, namely, Actigraphs [18, 19], and Metria-IH1 [18, 20] were chosen (Fig. 1a & 1b).
The goal of the study was to investigate the validity of estimates as provided by the respective SPAs from ActiLife and SenseWear. These SPAs are usually based on data from healthy individuals and the algorithms performance are optimized to the specific sensor locations. Therefore, the sensor locations for this study were chosen based on literature and the respective manufacturers prescription . This procedure was adopted to reduce any possible confounding factors to the outcome estimates that may arise due to switching of the sensor location from the optimal location for which the respective SPAs were developed. Furthermore, all wearable devices used in this study were obtained from the same manufacturing batch. This was done to minimize any measurement differences inherently arising due to variations in manufacturing process.
The ActiGraph wGT3X-BT’s  were worn on the upper arm , the waist [24, 25] and the ankle  (Fig. 1c). The waist sensor locations were chosen based on previous literature [13, 24]. For consistency, all the ActiGraphs were placed on the right side of the body. Adjustable fabric belts securely positioned the ActiGraphs on their respective locations. ActiGraphs measure the triaxial acceleration to estimate physical activity metrics . (Fig. 1a). The ActiGraphs sampled at 30 Hz.
Each ActiGraph device was assigned to a specific anatomical location. The device to anatomical location was held consistent between participants to minimize confounding factors due to unit calibration and sensor switching . The time on all the three devices were synchronized to the local atomic clock server time before data collection began.
Based on the manufacturer’s recommendation, the Metria-IH1 patch was adhered to the skin and located on the back of the upper left arm (Fig. 1b, c). The Metria IH1 patch houses variety of sensors to measure four modalities, namely, (i) 3-axis accelerometer, (ii) skin temperature, (iii) near body temperature and (iv) galvanic skin response (GSR). The module sampled data at the rate of 5000 data points per minute. The accelerometer alone sampled at 32 Hz. Metria-IH1 is a one-time use and throw device.
Three outcome metrics of relevance widely used in monitoring physical activity in clinical rehabilitation in individuals with stroke and iSCI were studied, namely, (i) step count, (ii) energy expenditure (EE) in Kcals, and (ii) metabolic energy (METs) [14, 28, 29]. These metrics were compared against the designated gold standard (Cosmed K4B2  for EE and MET and phone-based counter for step count).
Gold standard devices
Cosmed K4B2 (K4B2 Cosmed, Italy)  is a portable gas analysis system that measures oxygen consumption and Carbon-dioxide production in a breath by breath fashion. Extensive scientific usage and validation of Cosmed K4B2 makes it a gold standard in breath-by-breath metabolic measurements [22, 31]. For EE and MET metrics, the Cosmed K4B2 output was used as the gold standard comparison.
Experimental data collection
The experimental design and the test protocol are presented in Fig. 1(d) & (e). Consistent with recommended practice, the validation protocol was designed to cover a spectrum of physical activities . Data collection started with the subjects performing a series of activities such as lying, sitting, and standing for two-minutes each, a 50 steps walk on a hallway , a six minute walking test (6MWT) and finally two minutes of multi sit-to-stands. During multi sit-to-stand task, subjects were encouraged to do as many sit-to-stand sets as they safely and possibly can do.
Sufficient rests periods were given to the participants between each activity. The rest duration between activities were to ensure that the heart rate of the participants returned to their resting levels before starting the proceeding activity (refer Fig. 1(e)).
At the end of each activity, subjects self-reported their perceived effort of each activity using a Borg scale of perceived exertion . These self-reported ratings were later used to classify the intensity of activities performed based on the exertion levels. Participants were given the choice on whether they wanted to use their assistive device. All the participant’s completed the entire protocol successfully in a single visit. All the three devices, namely, the ActiGraphs, Metria and Cosmed K4B2 continuously and simultaneously recorded data during the entire session (i.e. 60 min).
For all data analysis, SPA provided by manufacturers of the respective devices were used for estimating the outcome metrics of interest (EE, MET and step count).
ActiLife:. The goal was to study the validity of outcome metrics while using SPAs in individuals with stroke and iSCI. The Choi  and Freedson  proprietary algorithms based off of healthy population empirical data was used to extract the EE and MET estimates for each of the activities in all of the groups. A Harris Benedict equation  was used to include the contribution of BMR to estimated EE and MET’s. This data extraction procedure was repeated for each of the wearable sensors positioned at the waist, ankle and the arm. The metrics from ActiGraph’s were used to study the effect of the sensor location, activity characteristics and population effects on the validity of outcome estimates.
Metria-IH1: The Metria-IH1’s proprietary fusion algorithm developed on the SenseWear software development kit platform, prompts for anthropometric information (age, height, weight, gender, dominant hand and smoking) at the time of data processing. This information is then fused along with information from the various on board sensor modules to calculate the outcome metrics of interest [20, 39]. The outcome metrics from Metria-IH1 were used to study the effect of activity characteristics and population effects on the validity of outcome estimates.
All the statistical analyses were performed using IBM SPSS 21.0 (SPSS Inc., Chicago, IL). Per study design, there were no direct between group comparisons (i.e. no direct comparison between healthy, iSCi and Stroke) and between device comparisons (i.e. no direct comparison between ActiGraph Vs Metria-IH1). For each sensor the postulated hypotheses were statistically compared with the designated gold standard’s estimate (i.e. EE & MET: Device vs. Cosmed K4B2; step count: device count vs. manual tally). The null hypothesis (H0) was that, the mean ranks of the groups (device estimates Vs Cosmed estimates) are the same. Therefore, failure to achieve the statistical significance, does not give sufficient evidence to reject the H0.
The statistical significance was set to p < .05 for all hypotheses tests pertaining to Metria-IH1. To account for multiple comparisons a corrected (Bonferroni) p value of p < .016 was used for all hypothesis testing pertaining to ActiLife estimates. For the group with stroke, data were analyzed based on the side of stroke impairment. (i.e. (Metria-IH1 for stroke (L) and ActiGraphs for stroke (R))).
Following epsilon-squared effect size (E2) thresholds were used for interpretation of strength of relationship: small effect size (E2 ≤ .1), medium effect size (.1 < E2 ≤ .3) and large effect size (E2 > .5) . With context to this analysis the following interpretation was used: a value of E2 = 1.0, indicated a large deviation and a value of E2 = 0 indicated a closer match in the mean rank of the estimated outcome with the designated golden standards (Cosmed K4B2 for EE and MET, manual step tally for step count).
Due to the smaller sample size and non-normal distribution of the data (based on Shapiro-Wilk test), non-parametric tests were used to statistically verify the postulated hypotheses. A series of Kruskal Wallis tests (Games-Howell multiple comparison for post-hoc analyses due to unequal variances verified using Levene’s test of equality for variances) were conducted to compare the mean rank and absolute agreement of outcome metrics estimated by each of the devices in comparison with Cosmed (designated gold-standard measurements . To verify step count validity, estimates of step count from ActiLife and Metria-IH1 SPAs were statistically compared with the manually counted steps during the 50 steps walk test.
For brevity only salient results are reported in the text. For detailed statistical results on bias between measures please refer to the Tables 1, 2 & 3. For detailed analysis of absolute agreement to assess reliability refer to the post-hoc analysis (additional file shows this in more detail [see Additional file 1; Supplementary Tables ST1 through ST8]). The results for the outcome metrics trended the same way from both the bias (Kruskal-Wallis) and absolute agreement tests (Games-Howell post-hoc).
In total, 28 participants completed the study. All participants completed the protocol successfully within 60 min. There were no adverse events. The descriptive statistics of the study population is provided in Table 1.
Activity type classification (Borg RPE)
Activities were classified as sedentary (lying, sitting and standing), low intensity (50 step walking) and high intensity (6MWT and multi sit-to-stand) (Table 1, Fig. 1(e)) based on the participants self-reported perceived exertion. Consistent with literature, individuals with stroke and iSCI rated their activities at a higher exertion level than healthy controls .
EE estimated by the ActiLife were significantly lower (p < .016; E2 > 0.7) than the Cosmed for all sedentary activities irrespective of the sensor locations and population studied. (Tables 2 & 3, Fig. 2a).
No significant differences were observed for EE estimates between Metria-IH1 and Cosmed for all sedentary activities in iSCI group (p > .05; E2 ≤ .3) and stroke groups with left impairment (n = 6) (p > .05; E2 < .3). In the healthy group, except for the EE during lying activity (p < .05; E2 > .7), no significant differences were observed between EE estimates from Metria-IH1 and Cosmed for other sedentary activities in the healthy controls (p > .05; E2 < .1); (Tables 2 & 3, Fig. 2b). The MET’s followed the same trend as EE’s.
These results show that the sensor type and characteristics of population studied can influence the accuracy of estimates during sedentary activity when using SPA.
Low intensity activity
No significant differences were observed between the manually counted steps (i.e. 50 steps) and the step counts estimated by the ActiLife (irrespective of sensor locations) and Senseware (Metria-IH1) for the healthy control group (p > 0.016) (Fig. 3). In the group with stroke, except the ActiLife estimates for ActiGraph located at the ankle (p > .016) (Fig. 3), ActiGrpah’s at all other locations and the Metria-IH1 significantly underestimated the step count (p < 0.016). For the group with iSCI, the ActiLife (all sensor locations) and Metria-IH1, significantly under-estimated the step counts (p < .016) (Fig. 3). These observations with step counts are consistent with previous literature, thus benchmarking the quality of data collected in this investigation [13, 42]. These results suggest that, (i) the step count estimates from standard algorithms can be influenced by effects such as sensor location and the optimal location to place sensor for step count tracking can vary depending on the specific type of population being studied.
The EE for the 50 step walk from the ActiLife for the healthy controls and iSCI groups were significantly different than the Cosmed when placed at arm and waist locations (p < 0.016) (Table 2, Fig. 2(a), 2(b)). No statistical significant differences were observed in the ActiLife EE estimates at all placement locations for the stroke group with right side impairment (p > .016) (Table 2, Fig. 2(c), 2(d)).
In general, the Metria-IH1 underestimated the EE for the low intensity activity for all groups (p < .05; medium effect sizes) [(χ2 = 7; E2 = .37)Healthy; (χ2 = 5.8; E2 = .39)iSCI; (χ2 = 6.5; E2 = .41)Stroke (L)] (Tables 2 & 3, Fig. 2(b)).
Overall the MET showed similar validity trends as EE for iSCI and stroke groups (Fig. 2). For the healthy group, except the MET from ActiLife at ankle, MET’s from all other locations showed same trend as EE.
These results highlight two main observations for low intensity activity; (1) sensor location that may be valid for estimating step count may not be valid for estimating metrics like EE/MET, and (2) using SPAs for estimating physical activity metrics in population with neurologic impairment like stroke and iSCI may yield inaccurate estimates.
High intensity activity
Six-minute walk test
In comparison to Cosmed estimates, no statistical significant differences were observed in the ActiLife EE estimates for the healthy controls at arm and waist locations (p > 0.016) (Table 2; Fig. 2(a)). However the ActiLife EE estimates from ActiGraph at ankle was significantly over estimated in comparison to cosmed (p < .016; E2 = .58) (Table 2). These results for the healthy group are in contrast to the EE trends from 50 step walk test. Thus, the activity intensity and duration can influence the accuracy of estimates from SPA even in a healthy group.
Similarly, for the 6MWT in the stroke group with right side impairment, in comparison to cosmed estimates, no significant difference was observed for the ActiLife EE estimates from ActiGraph’s at arm, waist and ankle (p > .016) (Table 2; Fig. 2(c)). However, for the stroke group with left side impairment, the EE estimates from ActiGraph at the right ankle, was only near significant with moderate effect size (p = 0.04; E2 = 0.39). Further, diminished effect sizes were observed for the ActiGraphs from the right side of the body for the group with left impaired. Similarly, it was observed that for the stroke group with right side impairment, the EE estimates from Metria-IH1 located at the left upper arm, was significantly different in comparison to Cosmed (p = 0.08; E2 = 0.43) (Table 2; Fig. 2(d)). This raises the possibility that when using fusion based sensor type to study EE (Metria-IH1- SenseWear), placing the sensors on the side of impairment may be a conservative approach for estimating EE. Indeed similar observations regarding side of impairment and EE estimate has been reported in stroke literature .
ActiLife EE estimates from all sensor locations for the group with iSCI, were significantly different (p < .016) in comparison to cosmed. The effect sizes ranging between medium to large (Tables 2 & 3, Fig. 2(a)). This shows that, the SPAs might not yield accurate results for EE in iSCI population. Based on effect size for EE estimates from 6MWT, the waist seems to be a non-optimal location for placing ActiGraph sensor for iSCI group. A potential reason for this could be the reduced walking speed in the iSCI group (0.5(0.22) m/s).
In all the groups studied, the Metria-IH1 EE estimates showed no statistical significant difference with Cosmed estimates (p > .05) [(χ2 = 1.0; E2 = .05)Healthy; (χ2 = .04; E2 = .00)iSCI; (χ2 = .03; E2 = .00)Stroke(L); (χ2 = 3.0; E2 = .43)Stroke (R)].
A rationale for Metria-IH1 to have performed well with iSCI sample studied despite using SPA based off of healthy controls could be that the SenseWear fuses a galvanic skin response (GSR) sensor information among others to estimate EE and MET. We speculate that the excess exertion during the prolonged 6MWT could have increased skin conductance due to increased sweating causing the Metria-IH1’s fusion SPA to over-estimate EE values and thus producing values closer to Cosmed. It is known that individuals with cervical iSCI are compromised in their autonomic nervous system functioning which could lead to reflex sweating . A majority (75%) of the iSCI group in this study had injury at cervical level where unilateral hyperhidrosis and reflex sweating is a reported phenomenon . Indeed, literature suggests that higher exertion could lead to higher skin conductance . The group with iSCI reported higher physical exertion during the low and high intensity activities (Table 1).
In comparison to Cosmed EE estimates, no significant differences were observed for the ActiLife EE estimates from ActiGraphs (Table 2; Fig. 2), (i) at arm and waist for healthy group, (ii) waist for the stroke groups and (iii) arm for the iSCI group. Irrespective of side of stroke impairment, the sensors at waist seems to be a desirable location to estimate EE during sit-to-stand task in stroke group. Except for the EE estimates for iSCI group, the Metria-IH underestimated the EE during the multi sit-to-stand task in stroke and healthy groups. The MET metrics followed a similar trend as the EE for healthy and stroke groups during 6MWT and multi sit-to-stand.
These results could suggest that, (i) the choice of sensor location could be dependent on activity type and outcome metric of interest and (ii) impairment conditions can significantly impact outcome metric estimated by SPA.
This study systematically analyzed the influence of four factors, namely, (i) choice of sensor type ((ActiGraph wG3TX-BT using ActiLife SPA) and Metria-IH1 -Senseware fusion based SPA), (ii) sensor location (ActiGraph wG3TX-BT at arm, waist and ankle and Metria-IH1 at arm) (iii) the activity characteristics and (iv) population effects (healthy, iSCI(ambulatory), Stroke) on the validity of three physical activity outcome metrics estimated by SPAs. Overall it was found that the physical activity metrics (EE, MET and step count) estimated by SPAs could be influenced significantly by these factors across the spectrum of activity levels studied.
Consistent with previous literature , the SPAs from both the sensors estimated the step count metric accurately in the control group, irrespective of sensor location (ActiLife) and type. However, observations from our data showed that, the estimates by these SPAs for EE and MET significantly diverged from the gold standard estimates at all activity levels. The sensor location, sensor type and activity type seemed to influence the EE and MET estimates provided by SPA even in healthy controls group. For instance, sensor placed at arm and waist seems to estimate EE and MET better during low and high intensity activities in comparison to sensor located at the ankle (Fig. 2(a)). Similar observations on healthy individuals have been reported in literature . Benchmarking with previous literature serves as a support for the consistency of data collected and analyzed in this study.
The SPAs estimation for the activity data collected from the study groups with iSCI and stroke were mediocre. For instance, in the group with iSCI, irrespective of sensor location and sensor type, the step counts estimates were inaccurate. Further, based on the ActiLife SPA estimates for iSCI, irrespective of sensor location and activity type studied, most of the EE estimates were significant deviants from the estimates produced by the designated gold standards. However, overall, based on a subjective comparison, the Sensewear performed relatively better for estimation in the iSCI and stroke groups . On the same lines, in comparison to ActiLife, the fusion algorithm from Senseware seemed to perform relatively better for EE and MET estimation during the studied sedentary tasks.
Finally, the trends from the multi sit-to-stand activity showed that, the sensor location should be chosen based on the nature of the activity type being studied. For instance, the arm and/or waist seems to be a desirable spot to estimate EE during sit-to-stand task for the healthy controls and the stroke group. For the iSCI group, sensors located at the arm seemed to capture the EE estimates well.
Overall, there are three possible reasons for such divergences observed in validity while using SPAs based off of healthy population to estimate outcomes from wearable sensors in stroke and iSCI.
Firstly, it is possible that the SPAs estimated outcome metrics based on the movement signature and acceleration thresholds empirically derived off of database collected from healthy population. These standard acceleration threshold values from healthy are far higher than those observed in neurological population [1, 2, 4, 49]. Individuals with iSCI and stroke in general walk with a gait speed much slower in comparison to healthy population [50, 51] and also use assistive devices. The average gait speeds during 6MWT for the different participant groups from our study sample were, 1.7(0.23) m/s, 0.5(0.22) m/s and 0.9(0.20) m/s for the healthy, iSCI and stroke respectively. Decreased speed and gait signatures while performing physical activities changes the acceleration thresholds leading to underestimates in step counts (an additional file shows this in more detail [see Additional file 2: Figure S1]). Secondly, sensors placed at different body locations (arm, waist and ankle) unfold different acceleration signatures (cut points) for a given activity type, due the dynamics constraint in motion between different body segments and sensor location [13, 52]. Finally, it is reasonable to expect that neurologic impairment like stroke and iSCI alters the metabolic profile. Hence empirically derived EE models based off of healthy controls may not work for this population [11, 53,54,55]. Thus the effect of choice of sensor type, sensor location and activity characteristics seems to be additional factor that needs consideration on top of population specific differences which influence the deviation of the estimates derived using SPAs.
We observed from our results that, for the stroke group, both the Actilife and Metria-IH1 (Senseware) SPAs performed relatively well when placed on the impaired side as opposed to unimpaired side for EE and MET estimation (Fig. 2(c), 2(d)). Overall, we also observed that despite using the standard SPAs some of the EE and MET estimates turned out to produce valid estimates for the group with stroke and iSCI.
As far as this sample data goes, there are a few possible explanations for above observations. One possible speculation from a pathophysiological angle could be the asymmetric sweat response that has been reported in individuals with stroke due to compromise in functioning of their autonomic nervous system [56,57,58].It is possible that the increased sweating on the paretic side could have improved the skin conductance, leading to the GSR sensor in Metria-IH1's (SenseWear) SPA overestimating the EE values, thus leading to values close to the Cosmed. Literature suggests that a higher physical exertion level can lead to higher skin conductance . Indeed self-reported physical exertion levels were higher for both the stroke and iSCI groups compared to healthy controls (Table 1). We speculate that a similar phenomenon could have led to overall better EE and MET estimates for the iSCI group while using Sensewear . However, it is promising to find support for this observation in literature . We did not record data regarding the sweat rate or skin temperature in this study. Nor did we have access to the Sensewear’s SPA to tease out the weightage given to GSR data in their fusion algorithm. These aspects require more work and specifically designed study to understand the influence of such factors are warranted.
We also suspect that since the participants from the stroke group in our study had mild-moderate gait impairments (mean gait speed was relatively higher 0.9 (0.20) m/s during 6MWT), the acceleration threshold was sufficient to create enough count points for the SPAs to produce better estimates. Similar observations have been noted in literature . We can only speculate that this could be the reason that the ActiLife SPAs estimates showed validity for some of the activities while used for the group with stroke.
Additionally from a sensor capability stand point, a potential reason for this could be that unlike the ActiLife (ActiGraph sampled at 30 Hz) the Metria-IH1’s SPA estimates EE and MET by fusing information from multiple on board sensor modules (overall sampling of all sensors at 5000 data points per minute in which accelerometer alone sampled at 32 Hz), such as, tri-axial accelerometer, near body skin temperature sensor and galvanic skin response in addition to customized participant specific information such as smoking behavior, and anthropometrics. Indeed literature shows that sensor fusion based approaches yield better measurements of metrics, contingent upon the quality of the sensors [2, 48, 59,60,61].
In summary, there were two main findings and recommendations form this pilot investigation, (i) the sensor type, sensor location, activity characteristics and the population studied influences the accuracy of estimation of physical activity metrics derived using SPAs: implementing advanced techniques like machine learning and data fusion to create customized population specific algorithms to estimate physical activity metrics in individuals with neurologic impairments such as iSCI and stroke has the potential to improve the reliability & accuracy and (ii) comprehensive validations including all outcome metrics (EE, MET and step counts) at different activity intensity level is recommended for validation of wearable sensors used in rehabilitation. These findings are also in consensus with findings from literature studying validity of wearable sensor estimates in different groups [13, 14, 52, 62,63,64,65].
Despite producing some novel and clinically useful information, this investigation has many limitations. The small sample size limits the extent of generalizability of our findings. However, the sample size in our study is comparable to other studies in this literature  and our findings were supported with observations from previous literature. The sedentary activities were recorded only for bouts of 2 min each. It is not clear if the same trends would hold when data is gathered for a different time scale. However, to justify our choice of this time scale, it is reasonable to assume that there is value in studying such small bouts (< 2 min) of sedentary activities as such low/high intensity activity occur through-out the day in community setting. Future studies with a larger sample size and includes other types of neurological impairments is recommended to explore the individual influence of each of the factors for population specific conditions on the outcome variables in laboratory as well as free living conditions.
On one hand the inferences and information from our results highlight the need to practice cautious decision making while choosing wearable sensor types and mounting locations for activity measurement in neurologic rehabilitation. Whilst on the other hand, implementing customized algorithms using advanced methods like machine learning and data fusion methodologies for estimating outcomes using wearables in individuals neurological impairments like stroke and iSCI could be beneficial . We maintain that, incorporating the combined effect of choice of sensor type used, location of placement, and the activity intensity being studied, to algorithms estimating outcome metrics from wearable devices may yield reliable physical activity metrics both in in-patient and out-patient environments.
Six-minute walk test
Incomplete spinal cord injury
Standard proprietary algorithm
Stroke with left impairment
Stroke with right impairment
Dobkin BH, Carmichael ST. The specific requirements of neural repair trials for stroke. Neurorehabil Neural Repair. 2016;30(5):470–78.
Dobkin BH, Dorsch A. The promise of mHealth: daily activity monitoring and outcome assessments by wearable sensors. Neurorehabil Neural Repair. 2011;25(9):788–98.
Matthews CE, Hagstromer M, Pober DM, Bowles HR. Best practices for using physical activity monitors in population-based research. Med Sci Sports Exerc. 2012;44(1 Suppl 1):S68–76.
Dobkin BH. Wearable motion sensors to continuously measure real-world physical activities. Curr Opin Neurol. 2013;26(6):602–8.
Webster KE, Wittwer JE, Feller JA. Validity of the GAITRite walkway system for the measurement of averaged and individual step parameters of gait. Gait Posture. 2005;22(4):317–21.
Chen S, Lach J, Lo B, Yang GZ. Toward pervasive gait analysis with wearable sensors: a systematic review. IEEE Journal of Biomedical and Health Informatics. 2016;20(6):1521–37.
Muro-de-la-Herran A, Garcia-Zapirain B, Mendez-Zorrilla A. Gait analysis methods: an overview of wearable and non-wearable systems, highlighting clinical applications. Sensors. 2014;14(2):3362.
Patel S, Park H, Bonato P, Chan L, Rodgers M. A review of wearable sensors and systems with application in rehabilitation. Journal of NeuroEngineering and Rehabilitation. 2012;9(1):1–17.
Gebruers N, Vanroy C, Truijen S, Engelborghs S, De Deyn PP. Monitoring of physical activity after stroke: a systematic review of accelerometry-based measures. Arch Phys Med Rehabil. 2010;91(2):288–97.
Haeuber E, Shaughnessy M, Forrester LW, Coleman KL, Macko RF. Accelerometer monitoring of home- and community-based ambulatory activity after stroke. Arch Phys Med Rehabil. 2004;85(12):1997–2001.
Moore SA, Hallsworth K, Plötz T, Ford GA, Rochester L, Trenell MI. Physical Activity, Sedentary Behaviour and Metabolic control following stroke: a cross-sectional and longitudinal study. PLoS One. 2013;8(1):e55263.
Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel M, Riley WT, Shar A, Spring B, Spruijt-Metz D, et al. Mobile health technology evaluation: the mHealth evidence workshop. Am J Prev Med. 2013;45(2):228–36.
Van Remoortel H, Giavedoni S, Raste Y, Burtin C, Louvaris Z, Gimeno-Santos E, Langer D, Glendenning A, Hopkinson NS, Vogiatzis I, et al. Validity of activity monitors in health and chronic disease: a systematic review. Int J Behav Nutr Phys Act. 2012;9(1):1–23.
Korpan SM, Schafer JL, Wilson KC, Webber SC. Effect of ActiGraph GT3X+ position and algorithm choice on step count accuracy in older adults. J Aging Phys Act. 2015;23(3):377–82.
Freedson P, Bowles HR, Troiano R, Haskell W. Assessment of physical activity using wearable monitors: recommendations for monitor calibration and use in the field. Med Sci Sports Exerc. 2012;44(1 Suppl 1):S1–4.
Giggins O, Clay I, Walsh L: Physical activity monitoring in patients with neurological disorders: a review of novel body-worn devices; 2017.
Lee JM, Kim Y, Welk GJ. Validity of consumer-based physical activity monitors. Med Sci Sports Exerc. 2014;46(9):1840–8.
Pantelopoulos A, Bourbakis NG. A survey on wearable sensor-based Systems for Health Monitoring and Prognosis. IEEE Transactions on Systems, Man, and Cybernetics. Part C (Applications and Reviews). 2010;40(1):1–12.
John D, Freedson P. ACTIGRAPH and ACTICAL physical activity monitors: a peek under the HOOD. Med Sci Sports Exerc. 2012;44(1 Suppl 1):S86–9.
Welk G, Kim Y, Shook RP, Ellingson L, Lobelo RL. Validation of a non-invasive, disposable activity monitor for clinical applications. J Phys Act Health. 2017;149(7):546-51.
Van Remoortel H, Giavedoni S, Raste Y, Burtin C, Louvaris Z, Gimeno-Santos E, Langer D, Glendenning A, Hopkinson NS, Vogiatzis I, et al. Validity of activity monitors in health and chronic disease: a systematic review. Int J Behav Nutr Phys Act. 2012;9(1):84.
John D, Freedson P. ActiGraph and Actical physical activity monitors: a peek under the hood. Med Sci Sports Exerc. 2012;44(1 Suppl 1):S86–9.
Bailey RR, Klaesner JW, Lang CE. An Accelerometry-based methodology for assessment of real-world bilateral upper extremity activity. PLoS One. 2014;9(7):e103135.
Aadland E, Ylvisåker E. Reliability of the Actigraph GT3X+ accelerometer in adults under free-living conditions. PLoS One. 2015;10(8):e0134606.
Feehan LM, Goldsmith CH, Leung AYF, Li LC. SenseWearMini and Actigraph GT3X accelerometer classification of observed sedentary and light-intensity physical activities in a laboratory setting. Physiother Can. 2016;68(2):116–23.
Webber SC, St John PD. Comparison of ActiGraph GT3X+ and StepWatch step count accuracy in geriatric rehabilitation patients. J Aging Phys Act. 2016;24(3):451–8.
Bassett DR, Rowlands AV, Trost SG. CALIBRATION AND VALIDATION OF WEARABLE MONITORS. Med Sci Sports Exerc. 2012;44(1 Suppl 1):S32–8.
Rothney MP, Schaefer EV, Neumann MM, Choi L, Chen KY. Validity of physical activity intensity predictions by ActiGraph, Actical, and RT3 accelerometers. Obesity (Silver Spring). 2008;16(8):1946–52.
Willey JZ, Moon YP, Paik MC, Boden-Albala B, Sacco RL, Elkind MS. Physical activity and risk of ischemic stroke in the northern Manhattan study. Neurology. 2009;73(21):1774–9.
Duffield R, Dawson B, Pinnington HC, Wong P. Accuracy and reliability of a Cosmed K4b2 portable gas analysis system. Journal of science and medicine in sport / Sports Medicine Australia. 2004;7(1):11–22.
de Lima C, Vancini RL, Arida RM, Guilhoto LM, de Mello MT, Barreto AT, Guaranha MS, Yacubian EM, Tufik S: Physiological and electroencephalographic responses to acute exhaustive physical exercise in people with juvenile myoclonic epilepsy. Epilepsy Behav. 2011, 22(4):718–722.
Lee JA, Laurson KR. Validity of the SenseWear armband step count measure during controlled and free-living conditions. Journal of Exercise Science & Fitness. 2015;13(1):16–23.
Dijkstra B, Zijlstra W, Scherder E, Kamsma Y. Detection of walking periods and number of steps in older adults and patients with Parkinson's disease: accuracy of a pedometer and an accelerometry-based method. Age Ageing. 2008;37(4):436–41.
Godfrey A, Del Din S, Barry G, Mathers JC, Rochester L. Instrumenting gait with an accelerometer: a system and algorithm examination. Med Eng Phys. 2015;37(4):400–7.
Borg GA. Psychophysical bases of perceived exertion. Med Sci Sports Exerc. 1982;14(5):377–81.
Choi L, Liu Z, Matthews CE, Buchowski MS. Validation of accelerometer wear and nonwear time classification algorithm. Med Sci Sports Exerc. 2011;43(2):357–64.
Freedson PS, Melanson E, Sirard J. Calibration of the computer science and applications, Inc. accelerometer. Med Sci Sports Exerc. 1998;30(5):777–81.
Roza AM, Shizgal HM. The Harris Benedict equation reevaluated: resting energy requirements and the body cell mass. Am J Clin Nutr. 1984;40(1):168–82.
Jayaraman C, Mummidisetty CK, Jayaraman A. Effect of wearable sensor dynamics on physical activity estimates: a comparison between SCI vs. healthy individuals. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC): 16–20 Aug 2016; 2016. p. 3282–5.
Cohen J. Statistical power analysis for the behavioral sciences. Routledge Academic: NewYork, NY; 1988.
Dobkin BH. Fatigue versus activity-dependent fatigability in patients with central or peripheral motor impairments. Neurorehabil Neural Repair. 2008;22(2):105–10.
Klassen TD, Simpson LA, Lim SB, Louie DR, Parappilly B, Sakakibara BM, Zbogar D, Eng JJ. “Stepping up” activity Poststroke: ankle-positioned accelerometer can accurately record steps during slow walking. Phys Ther. 2016;96(3):355–60.
Manns PJ, Haennel RG. SenseWear armband and stroke: validity of energy expenditure and step count measurement during walking. Stroke Research and Treatment. 2012;2012:8.
Fast A. Reflex sweating in patients with spinal cord injury: a review. Arch Phys Med Rehabil. 1977;58(10):435–7.
Gorman PH. Unilateral hyperhidrosis from a contralateral source in an individual with C4 complete tetraplegia. The journal of spinal cord medicine. 2010;33(4):428–30.
Altini M, Penders J, Vullers R, Amft O. Combining wearable accelerometer and physiological data for activity and energy expenditure estimation. In: Proceedings of the 4th conference on wireless health. Baltimore, Maryland: ACM; 2013. p. 1–8.
Parkka J, Ermes M, Antila K, Mv G, Manttari A, Nieminen H. Estimating intensity of physical activity: a comparison of wearable accelerometer and gyro sensors and 3 sensor locations. In: 2007 29th annual international conference of the IEEE engineering in medicine and biology society: 22–26 Aug. 2007 2007; 2007. p. 1511–4.
Liu S, Gao RX, John D, Staudenmayer JW, Freedson PS. Multisensor data fusion for physical activity assessment. IEEE Trans Biomed Eng. 2012;59(3):687–96.
Nero H, Benka Wallen M, Franzen E, Stahle A, Hagstromer M. Accelerometer cut points for physical activity assessment of older adults with Parkinson's disease. PLoS One. 2015;10(9):e0135899.
Dickstein R. Rehabilitation of gait speed after stroke: a critical review of intervention approaches. Neurorehabil Neural Repair. 2008;22(6):649–60.
Kim CM, Eng JJ, Whittaker MW. Level walking and ambulatory capacity in persons with incomplete spinal cord injury: relationship with muscle strength. Spinal Cord. 2004;42(3):156–62.
Altini M, Penders J, Vullers R, Amft O. Estimating energy expenditure using body-worn accelerometers: a comparison of methods, sensors number and positioning. IEEE journal of biomedical and health informatics. 2015;19(1):219–26.
Billinger S. Cardiovascular regulation after stroke: evidence of impairment, trainability, and implications for rehabilitation. Cardiopulmonary Physical Therapy Journal. 2010;21(1):22–4.
Buchholz AC, Pencharz PB. Energy expenditure in chronic spinal cord injury. Curr Opin Clin Nutr Metab Care. 2004;7(6):635–9.
Potempa K, Braun LT, Tinknell T, Popovich J. Benefits of aerobic exercise after stroke. Sports Med. 1996;21(5):337–46.
Al-Qudah ZA, Yacoub HA, Souayah N. Disorders of the autonomic nervous system after hemispheric cerebrovascular disorders: an update. Journal of Vascular and Interventional Neurology. 2015;8(4):43–52.
Sakashita Y, Kakuta K, Kakuma K, Matsuda H. Unilateral persistent hyperhidrosis after ischemic stroke. Rinsho shinkeigaku = Clinical neurology. 1992;32(4):454–6.
Korpelainen JT, Sotaniemi KA, Myllyla VV. Asymmetric sweating in stroke: a prospective quantitative study of patients with hemispheral brain infarction. Neurology. 1993;43(6):1211–4.
King RC, Villeneuve E, White RJ, Sherratt RS, Holderbaum W, Harwin WS. Application of data fusion techniques and technologies for wearable health monitoring. Med Eng Phys. 2017;42:1–12.
Hiremath SV, Ding D, Farringdon J, Cooper RA. Predicting energy expenditure of manual wheelchair users with spinal cord injury using a multisensor-based activity monitor. Arch Phys Med Rehabil. 2012;93(11):1937–43.
Albert MV, Azeze Y, Courtois M, Jayaraman A. In-lab versus at-home activity recognition in ambulatory subjects with incomplete spinal cord injury. Journal of NeuroEngineering and Rehabilitation. 2017;14(1):1–6.
Gastin PB, Cayzer C, Dwyer D, Robertson S. Validity of the ActiGraph GT3X+ and BodyMedia SenseWear armband to estimate energy expenditure during physical activity and sport. J Sci Med Sport. 2018;21(3):291-295. https://doi.org/10.1016/j.jsams.2017.07.022. Epub 2017 Jul 31.
Coote S, O'Dwyer C. Comparative validity of accelerometer-based measures of physical activity for people with multiple sclerosis. Arch Phys Med Rehabil. 2012;93(11):2022–8.
Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. The International Journal of Behavioral Nutrition and Physical Activity. 2015;12:159.
Giggins OM, Clay I, Walsh L. Physical activity monitoring in patients with neurological disorders: a review of novel body-worn devices. Digital Biomarkers. 2017;1(1):14–42.
Saremi K, Marehbian J, Yan X, Regnaux JP, Elashoff R, Bussel B, Dobkin BH. Reliability and validity of bilateral thigh and foot accelerometry measures of walking in healthy and hemiparetic subjects. Neurorehabil Neural Repair. 2006;20(2):297–305.
We thanks our participants for volunteering to take part in the study.
This pilot study was funded in part by the Max Näder Lab for Rehabilitation Technologies and Outcomes Research at Shirley Ryan Abilitylab. The funding bodies had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Availability of data and materials
Please contact author for data requests.
Ethics approval and consent to participate
All the study procedures were approved by the authorized local institutional review board, Northwestern University. Upon arrival, all the subjects provided informed consent. Decision to participate in the study was voluntary.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
tables with post-hoc analysis. Description:Table ST1. Games-Howell multiple comparison post-hoc test to assess absolute agreement (EE estimates in healthy control group). Table ST2. Games-Howell multiple comparison post-hoc test to assess absolute agreement (MET estimates in healthy control group). Table ST3. Games-Howell multiple comparison post-hoc test to assess absolute agreement (EE estimates in iSCI control group). Table ST4. Games-Howell multiple comparison post-hoc test to assess absolute agreement (MET estimates in iSCI control group). Table ST5. Games-Howell multiple comparison post-hoc test to assess absolute agreement (EE estimates in stroke group with right impairment). Table ST6. Games-Howell multiple comparison post-hoc test to assess absolute agreement (MET estimates in stroke group with right impairment). Table ST7. Games-Howell multiple comparison post-hoc test to assess absolute agreement (EE estimates in stroke group with left impairment). Table ST8. Games-Howell multiple comparison post-hoc test to assess absolute agreement (MET estimates in stroke group with left impairment). (DOCX 41kb)
FigureS1. Sample acceleration data. Acceleration (triaxial) from ActiGraphs strapped to arm, waist and ankle during a 50 step walk test from a representative healthy, iSCI and stroke participants. Note: The duration for the 50 step walk test is different across the groups. (PNG 93 kb)
About this article
Cite this article
Jayaraman, C., Mummidisetty, C., Mannix-Slobig, A. et al. Variables influencing wearable sensor outcome estimates in individuals with stroke and incomplete spinal cord injury: a pilot investigation validating two research grade sensors. J NeuroEngineering Rehabil 15, 19 (2018). https://doi.org/10.1186/s12984-018-0358-y
- Wearable devices
- Step counts
- Energy expenditure
- Metabolic equivalent
- Spinal cord injury
- Sweat rate