Skip to main content

Do children and adolescent ice hockey players with and without a history of concussion differ in robotic testing of sensory, motor and cognitive function?



KINARM end point robotic testing on a range of tasks evaluating sensory, motor and cognitive function in children/adolescents with no neurologic impairment has been shown to be reliable. The objective of this study was to determine whether differences in baseline performance on multiple robotic tasks could be identified between pediatric/adolescent ice hockey players (age range 10–14) with and without a history of concussion.


Three hundred and eighty-five pediatric/adolescent ice hockey players (ages 10–14) completed robotic testing (94 with and 292 without a history of concussion). Five robotic tasks characterized sensorimotor and/or cognitive performance with assessment of reaching, position sense, bimanual motor function, visuospatial skills, attention and decision-making. Seventy-six performance parameters are reported across all tasks.


There were no significant differences in performance demonstrated between children with a history of concussion [median number of days since last concussion: 480 (range 8–3330)] and those without across all five tasks. Performance by the children with no history of concussion was used to identify parameter reference ranges that spanned 95 % of the group. All 76 parameter means from the concussion group fell within the normative reference ranges.


There are no differences in sensorimotor and/or cognitive performance across multiple parameters using KINARM end point robotic testing in children/adolescents with or without a history of concussion.


The rate of child and adolescent participation in organized sport is high, which has significant health benefits related to regular exercise. However, youth sustain sport-related concussions, accounting for more than 15 % of all injuries in 9–16 year old players [13]. Concussion is a brain injury and has been defined as a complex pathophysiological process affecting the brain, induced by biomechanical forces [4]. In general, the majority (80–90 %) of concussions resolve in a short (7–10 days) period [5, 6]. Our understanding of the impact of concussion(s) on the brain is limited, however neuropsychological deficits have been observed in adults over a time span ranging from 24 h to 3 years [715]. Within the last decade research related to concussion in children and adolescents has rapidly expanded [1618].

Of particular interest is the number of sport-related concussions sustained while playing ice hockey, which is popular in Canada and the USA with about 850,000 children playing in both countries [1, 2]. There is growing concern regarding the impact of concussion in this population [3, 1622]. For example in Alberta, Canada, overall concussion injury rates [based on the number of injuries per 1000 player-hours (95 % confidence interval)] have been shown to range from 0.79 (0.55 to 1.13) to 2.73 (1.90 to 3.94) [2123]. Researchers from London, Ontario, Canada examined a retrospective cohort of children/adolescents (<18 years of age) attending the emergency department who had sustained concussions (2006 to 2011). They demonstrated that 36 % of youth that sustained a sport-related concussion did so while playing ice hockey [24]. Evidence suggests that children and adolescents may be more susceptible to concussion, and may take longer to recover than adults [1618, 25]. The impact of sport related concussion(s) on motor and cognitive processing in children, with respect to the effect on the developing brain, is poorly understood [26, 27].

The injury spectrum associated with concussion is broad, ranging from subtle or imperceptible to obvious changes in motor and/or cognitive performance, and very dependent on the developmental stage of the central nervous system (CNS) [2831]. One of the primary reasons for the paucity of research related to the effect of concussion in children and adolescents is the lack of sensitive measurement tools that can identify impairments following concussion [32, 33]. Better diagnostic and prognostic tools are needed to address issues related to early diagnosis and management of concussion across the continuum of aging but particularly in children and adolescents. Maturation occurs at different rates across various domains within the CNS, ranging broadly from 18 years of age (reaching correction) to 30 (precision of number sense), which can complicate concussion evaluation in children and adolescents [3436]. Researchers are beginning to examine the efficacy of different measurement tools used with adults among children and adolescents [37, 38].

Robotic technology has the potential for use as a clinical diagnostic assessment tool as it is ideal for objective, quantitative, rapid and automated assessment of neural function [39, 40]. Further, robots have often been used as treatment tools for individuals with brain damage [4145]. The KINARM exoskeleton (BKIN Technologies Ltd, Ontario, Canada) is a robotic device that has been used to detect functional impairments across neurological domains in adults [40, 4652]. Various tasks test visuomotor skills, proprioceptive function, rapid decision making, and executive function capabilities [4652]. The KINARM end-point robot has been used to examine neurologic impairments in adult subjects post-concussion [52, 53]. The results from one study identified subjects with post-concussion syndrome (symptoms of the concussion that persist for weeks or months) had more abnormal scores than those without post-concussion syndrome [52, 54, 55]. There is evidence that the KINARM exoskeleton robot is reliable and sufficiently sensitive to use in adult stroke and moderate/severe brain injury populations [40, 4651]. The KINARM end point robot also shows both relative reliability (intra-class correlation coefficients) and absolute reliability (Bland-Altman agreement) among healthy boys with no neurological impairment, who range in age from 10 to 14 [56].

The primary objective of the current study was to evaluate differences in KINARM end point robotic testing outcomes (sensorimotor and cognitive) between children/adolescent ice hockey players with and without a history of concussion. Comparisons in performance were made between those with a history of concussion [median number of days since last concussion: 480 (range 8–3330)] and those without. Reference ranges were determined for the robotic tests based on data from the controls (no history of concussion). Further, the reported history of concussion on subject performance was examined relative to the parameters of the five KINARM tasks.



The study is an observational cross-sectional design. Three hundred and eighty-five healthy children/adolescents (males and females) completed KINARM end point robot testing: 94 with a history of concussion and 292 without. Inclusion criteria for the present study were: ice-hockey players, ages 10–14, with no current neurological symptoms, and with and without a prior traumatically induced transient disturbance of the brain [57]. Subjects were excluded if they had a significant orthopedic injury (i.e. fracture) in the upper extremity, visual impairment (<20/50 corrected) or had sustained a concussion less than 5 days prior to testing. Concussion history was obtained by self-report through the use of a questionnaire (adapted from a similar questionnaire used in youth hockey studies previously conducted in this laboratory) that children completed, with the help of their parents [21]. Subjects were asked to record the number of previously sustained concussions (whether diagnosed by a physician) as well as the date and activity participating in at the time of the concussion. The most recent concussion referenced was used for the analysis in the current study. This was a sample of convenience. The Conjoint Health Research Ethics Board at the University of Calgary approved the study (Ethics ID number E24026). Prior to data collection, a parent or guardian provided signed consent for the subject to participate in all aspects of the study and the children provided assent. Parental consent was provided for the photograph in Fig. 1 to be included in the current publication.

Fig. 1

The KINARM end point robot. The virtual reality workstation projects targets onto a screen

Robotic assessment

At the beginning of the 2013–2014 ice hockey season players from Pee Wee ice hockey teams (ages 10–14) completed KINARM end point robot testing at the University of Calgary. The robotic assessment was performed using the KINARM end point bimanual device (BKIN Technologies Ltd, Kingston, Ontario, Canada), which permits free movement of the upper extremities in the horizontal plane while seated (refer to Fig. 1). A virtual reality system displays visual targets such that they appear in the same plane as the arms. Subjects experience force feedback while grasping the robot handles when hitting targets during specific tasks. The protocol included assessments of reaching, position sense, bimanual motor function, visuospatial skills, attention, and decision-making [56].

The testing session lasted approximately 15 min, which included seating the subject at the robot, instructions and completion of the five tasks. Subjects were seated in a chair in front of the robot, asked to avoid slouching, and the robot height adjusted such that each child’s head rested on a location in the center of the visual field. Body position was kept constant across subjects. Subjects completed the following five tasks during each testing session: Visually Guided Reaching (VGR) on right and left, Arm Position Matching (APM) on right and left, Object Hit (OH), Object Hit and Avoid (OHA), and Trail Making B (TMB) with the dominant limb [56]. These tasks characterize sensorimotor and/or cognitive performance. Seventy-six parameters from the five tasks are presented in the paper.

Experimental tasks

Visually guided reaching task

This task provides a measure of upper extremity visuomotor capability (Fig. 2a) [56]. The robot handle is represented as a white dot (0.5-cm radius) on the display. The task targets are red circles, each with a 1.0-cm radius. Subjects reach out and back between the central and destination targets. Four red targets are 10 cm from the initial central target. Subjects are asked to move the white dot from the centre of one target to the centre of the next target that appears, as quickly and accurately as possible. All targets are located near the centre of the workspace for each arm. There are five blocks of trials, target location is randomized within a block and both the reach out and reach back trials are analyzed. This process is repeated forty times to explore the workspace and measure variability of the subject’s responses. Each subject completed the task twice, once with each arm; the dominant arm always preceded the non-dominant arm. Although not identical, the task used in the current work is similar to and uses metrics that were described earlier using the KINARM exoskeleton robot [4648, 50, 56].

Fig. 2

The five KINARM robot tasks used in the study. a Visually guided reaching with the right arm, b Arm position matching with the right arm, c Object hit, d Object hit and avoid, and e. Trail making B (not to scale, example of the alpha-numeric alternation) This figure has been modified from a previous paper [56]

Arm position matching task

This task provides a measure of proprioceptive (position sense) capability (Fig. 2b) [56]. The robot moves one arm (passive arm) to one of four different target locations spaced at the corners of a square grid at 20 cm intervals in the X and Y directions. Movements are made with a bell-shaped velocity profile. Subjects are then instructed to actively move the opposite arm (active arm) to the mirror location of the passive arm in space. Subjects notified the examiner once the mirror-matched position was reached and the examiner advanced the robot to the next trial. The subject’s vision is blocked to ensure that any sensory information about limb position is from proprioceptive inputs. There are six blocks of trials, target location is randomized within a block and one trial for each target is completed within a block. The same target is never repeated sequentially. The task was completed twice with dominant arm being the active arm first followed by the non-dominant arm. A similar task has been used with the KINARM exoskeleton robot; to reduce time for the assessment of subjects four targets were used rather than nine [4648, 50, 56].

Object hit task

This task is a rapid sensorimotor, decision and control test (Fig. 2c) [56]. It assesses the ability of a subject to select and engage motor actions with both hands over a range of speeds and a large workspace. Virtual paddles appear at the robot handles. Subjects are asked to use the paddles to hit balls that appear to fall from the top of the screen toward them. The robot produces a reactive force that mimics the actual force that would have been felt by the subject if these were real objects contacting a real paddle (perceived as having physical properties similar to a squash ball). As the task proceeds the balls move at greater speeds and appear more often, making the task more difficult as time progresses. Balls fall at random from ten bins that are spread equally across the workspace, 30 balls fall from each bin. A total of 300 balls are dropped during the task in just over one minute. A similar task has been used with older adults and the KINARM exoskeleton robot, except the version used in the current work used smaller paddles (2 cm rather than 5 cm), the balls moved faster and more balls could be present on the screen at a given time [49, 56].

Object hit and avoid

This task is similar to the OH task, but requires more cognitive control. Subjects must hit specific target objects while avoiding all others (Fig. 2d) [56]. The emphasis is on attention, rapid motor selection, and inhibition to avoid distractor targets. At the start of the task subjects are shown two target shapes out of a possible eight, they are instructed to memorize these as the only two shapes to hit during the task, and to avoid all other (6) distractor shapes. If distractors hit the subject’s paddles they pass through the paddles but there is no reactive force felt by the subject. This provides immediate and ongoing feedback to the subject that the object was a distracter and not a target. As with the preceding task, when targets are hit the robot produces a reactive force that mimics the actual force that would have been felt by the subject if these were real objects contacting a real paddle. A similar task has been used with older adults and the KINARM exoskeleton robot, except the version used in the current work used smaller paddles (2 cm rather than 5 cm), the objects moved slightly faster and more objects could be present on the screen at a given time [56].

Trail Making B task

This task is the second part of a cognitive test that evaluates executive function (e.g. visual attention and task switching) from the field of neuropsychology that is commonly used in the assessment of brain injury (Fig. 2e) [38, 56]. Participants trace through an alternating alpha-numeric sequence of targets 1-A-2-B for example, up to 13, for a total of 25 targets. A shortened version of the task, that has only five sequential targets preceded the full task to help familiarize subjects with the task. If the subject touches an incorrect target while moving through the sequence the preceding correct target will turn red and the subject must return to that target before continuing. The task pattern was randomly selected from eight possible patterns [Matlab (Mathworks, Natick, MA, USA)] for the TMB task [56, 58].

Outcome measures

Parameters for each task have been previously developed to quantify task performance in healthy individuals and those with brain damage, thus behavioral attributes associated with the parameters are included in Table 1. In summary, the visually guided reaching task included 10 parameters: posture speed (m/s), reaction time (s), initial direction error (rad), initial distance ratio, initial speed ratio, speed maxima count, minimum maximum speed difference (m/s), movement time (s), path length ratio, and max speed (m/s). The arm position matching task included 12 parameters: variability X, Y, and XY (m), contraction/expansion ratio X, Y, and XY, shift X, Y, and XY (m), absolute error X, Y, and XY. The object hit task included 14 parameters: total hits, hits with left, hits with right, hand bias hits, miss bias, hand transition, hand selection overlap, median error, hand speed left and right (m/s), movement area left and right (m2), and movement area bias. Object hit and avoid include the same parameters as outlined for object hit (hand bias hits, hand speed bias, and movement area bias were removed from the primary analysis) with three additions: distractor hits left, right and total. Trail making task B included the following four parameters: total time (s), dwell time (s), time ratio and error count.

Table 1 Summary of the five KINARM robot tasks, including behavioral attributes and definitions of task parameters

Data analysis

Performance by the group with no history of concussion was used to identify reference ranges for each parameter that spanned 95 % of the group [47]. Most of the time, the 95 % range was one-sided, reflecting the fact that abnormal values would be expected to be larger or smaller than the comparison sample (e.g. reaction time would be expected to be slower in individuals with concussion). Mean values of subjects with a history of concussion were calculated and then assessed to identify if they fell within the reference ranges.

All subjects and their data were included in the analysis as there were no missing data points. Individuals with a history of concussion [median number of days since last concussion: 480 (range 8–3330)] were further subdivided. Two separate grouping strategies are presented in an attempt to fully explore the relationship of timing since concussion to performance of those children with a history of concussion as related to the reference range developed from the performance of children with no history of concussion. In grouping A, the data from children with a history of concussion was divided into three equal terciles based on time since their last concussion to determine whether timeline since concussion would impact parameter performance. Terciles were chosen to ensure that an equivalent number of children were present in each of the three bins. The timelines associated with each tercile are as follows: T1 = 8–330 days (0–11 months), T2 = 331–990 days (11–33 months) and T3 = 991–3330 days (33–111 months). The time since their most recent concussion was obtained from a questionnaire (adapted from a similar questionnaire used in youth hockey studies previously conducted in this laboratory) that children filled out at home with parental input and submitted to the researchers at testing [21]. In grouping B, we were specifically interested in those cases that had been concussed in the last year. We divided the data according to five specific time points post-concussion: 3 weeks, 1, 3, 6 months, and 1 year.

Statistical analyses were performed in SPSS version 21 [59]. The significance level was set at alpha = 0.05. Since eight different patterns were randomly selected across subjects for the TMB task, a preliminary analysis was performed to ensure that parameter outcomes were not affected by TMB task version. Total time (s) was selected as the representative dependent variable in the TMB task analysis and was performed on the group of subjects with no history of concussion. Output from a one-way analysis of variance (ANOVA) showed no statistically significant difference in performance across the eight versions of the TMB tasks used during testing, F(7, 28) = 1.748, p = 0.098, thus parameters for each version of the TMB task were collapsed in order to perform the primary analysis for the TMB task.

The primary analysis in the current study included separate multivariate analyses of variance (MANOVA) for each of the five robot tasks, with right and left side analyzed separately for the Visually Guided Reaching and Arm Position Matching tasks. Those children/adolescents with and without a history of concussion were compared to determine differences in performance. Separate exploratory analyses were performed using MANOVA for each of the five robot tasks to evaluate if differences existed between age groups (Ages 10 and 11 versus Age group 12,13,14) and for each of the five robot tasks to evaluate if differences existed between Handedness groups (Right versus Left). As stated above right and left side were analyzed separately for the Visually Guided Reaching and Arm Position Matching tasks. One-way analysis of variance (ANOVA) was used to identify differences across individual parameters when significant group effects were found.


Characteristics of the subjects who took part in the study are found in Table 2. The primary analysis showed no significant difference in performance, based on parameter values, between those children/adolescents with and without a history of concussion across all five tasks (refer to Table 3). Table 4 presents a summary of the outcomes from the secondary exploratory analysis from separate MANOVAs that compared Age Groups and Handedness. The results from the MANOVA exploratory analysis showed that there was a difference in at least one of the variables between Age Groups, while the ANOVAs identified which parameters were significantly different: Visually Guided Reaching (R & L), Object Hit, Object Hit and Avoid, and TMB task. Furthermore, the MANOVA analysis also showed a difference in at least one of the variables related to Handedness, while the ANOVAs also identified which parameters were significantly different: Visually Guided Reach (L only), Arm Position Matching (R only), and Object Hit.

Table 2 Summary of the study population characteristics
Table 3 Primary statistical analysis outcomes
Table 4 Exploratory statistical analysis outcomes

The reference ranges (based on performance from participants with no history of concussion) for parameters for all five KINARM end point robot tasks are presented in Table 5 and this includes identification of the direction of failure related to performance. Visual inspection was used to determine whether the mean values for parameters from children/adolescents with a history of concussion fell within the reference ranges (refer to Table 6). For grouping A of the data, the parameter mean values for all subjects in each of the terciles (time since most recent concussion) also fell within the established reference ranges (refer to Table 7). With respect to grouping B, due to the variability in the timelines among the children the mean and range about each of these time points are as follows: 3 weeks [0.5 months (8–21 days)], 1 month [1.5 months (30–60 days)], 3 months [3.5 months (90–120 days)], 6 months [7 months (150–270 days)], 1 year [11 months (300–360 days)]. An unequal number of children are present in each of the five bins: 0.5 months (2), 1.5 months (9), 3.5 months (3), 7 months (9), and 11 months (13). Refer to Fig. 3 for the frequency of parameter failures across the five time point bins. Figure 3 represents the frequency that all subjects with a past history of concussion fell outside the 95 % range of controls relative to the time since their most recent concussion. The occurrence of this was rare considering at each time point there were 36 subjects and 76 parameters (e.g. The total potential parameter failure is 2736).

Table 5 Parameter reference range established from healthy controls
Table 6 Parameter reference range for healthy controls versus children with concussion history
Table 7 Parameter reference range versus terciles based on time since last concussion
Fig. 3

Title: Parameter failure versus time since most recent concussion. Represents the frequency that all subjects with a past history of concussion fell outside the 95 % range of controls relative to the time since their most recent concussion. The occurrence of this was rare considering at each time point there were 36 subjects and 76 parameters (i.e. The total potential parameter failure is 2736)


The main purpose of this study was to compare performance between children/adolescents with and without a history of concussion across five different robotic tasks, variations of which have previously been used to measure neurologic deficits in individuals with a variety of types of brain damage [40, 44, 4651]. These tasks are designed to assess reaching, position sense, bimanual motor function, visuospatial skills, attention, and decision-making. The key finding is that the history of concussion group showed no difference in performance as compared to those subjects who reported no history of concussion.

The KINARM robot has specifically been used to evaluate Visually Guided Reaching and Arm Position Matching tasks in adults with mild to severe traumatic brain injury [47, 52]. In the current study, it is entirely possible that any neural impairment associated with concussion was resolved by the time the individuals were examined. In an attempt to address this issue subjects with a history of concussion were subdivided into two separate groupings (A and B) based on time since concussion. In the groups we examined we could find little to no difference in the presence of impairments on robotic testing based on timing.

Another consideration is that the impact of concussion on the elements of the nervous system examined in this study, if persistent, was so small that it fell well within the reference range recorded from individuals without a history of concussion. A limitation of the current study is the fact that individual subjects were not examined, only group effects. It could be that impairments were present on an individual basis, yet insufficient to cause a group effect. If a majority of the individuals with a history of concussion are truly symptom free this will mask the influence of the few individuals with real problems. Another limitation of the study is the fact that concussion history was obtained through self-report (children and parents). The accuracy of self-reported data on medical history is influenced by several factors such as the patient’s knowledge and understanding of the relevant information, ability to recall it, and willingness to report it [59]. Although the literature suggests there are problems with the reliability of self-report, self report measures are preferred as they are cost-effective and time efficient relative to physical examinations and lab testing particularly in large study groups [6064]. In the current study there was simply no other way to conduct the study.

Another limitation of the study is that it is cross-sectional in design. Due to the nature of the fact that children were at various points from their concussion, this effectively increases the heterogeneity in our study. Prospective studies will be helpful moving forward. In this design, individuals can be evaluated before and after concussion and act as their own control. The heterogeneous nature of maturation within the central nervous system in this population however may result in increased levels of variability [65].

Due to the fact that, in many cases, there was a significant time lag between the most recent concussion and when we conducted the study, one could argue that this introduced a recall bias. However, this would be an equivalent issue in both groups as recall bias is non-differential, misclassification related to recall can occur in either group. It could also be argued that individuals in the no history of concussion group could be misclassified due to an undiagnosed concussion which has been raised in the literature related to impact sports such as ice-hockey and football [66].

Despite having a measurement tool that is reliable, accurate and precise, as mentioned previously, there can still be substantial variability across normal human behavior. This type of variability is something that can be seen in many biomarkers used in medicine and is simply one of the limitations of any tool that relies on a normative reference range [67]. One subject’s “normal” may represent a slight impairment for another individual. It may explain those “parameter failures” identified in subjects evaluated using grouping B. This could be dealt with by baseline testing all athletes and having individuals return when they sustain a concussion. In general children and adolescents have been shown to take longer to recover following concussion than their adult counterpart’s [15, 17].

The alternatives currently available to measure neurologic impairment following brain injury are prone to rely on observer-based ordinal scales or symptom questionnaires, each of which has their own set of limitations. For example, clinical symptom ratings are the foundation of most concussion management protocols [66]. However, many factors can affect symptom reporting in youth and adult athletes [68]. Symptom reporting can be influenced by a number of clinical, demographic and methodological variables. For example, post-concussion-like symptoms are nonspecific and are reported by uninjured athletes, healthy community-dwelling adults, and people with chronic pain [6973]. In university settings, women have been shown to identify more symptoms on baseline testing than men [74]. In one study a subgroup of the study population reported no increase in symptoms post-concussion even though degradation in cognitive performance was identified on cognitive (ImPACT) testing [75].

Lastly, one might consider that concussion does not impact performance on the five tasks used in the present study. Preliminary evidence in elite athletes (mean age: 21 years) would suggest otherwise [53]. That study made comparisons between athletes at baseline and then again <72 h post-concussion using the same tasks used in the present study [54]. Over a period of days and weeks, those athletes returned to baseline [69]. In another study in adults with mild traumatic brain injury, conducted in the Emergency room in a Cincinnati (Ohio) hospital, four or more abnormal parameter scores were identified related to the Arm Position Matching task when compared to previously published normative reference data [47, 52]. Given the above findings we argue that, within the range of the neurologic impairments assessed these children have no persistent significant effects of their concussion.

The results from the exploratory analysis suggested a significant difference in performance between age groups in the following tasks: Visually Guided Reaching (for both the right and left upper extremity), Object Hit, Object Hit and Avoid, and Trail Making B tasks. The exploratory analysis also suggested a significant difference between upper extremity dominance (handedness) for the following: Visually Guided Reaching (left upper extremity) and Object Hit tasks. Age groups and handedness would be better suited to be covariates as part of multivariate analyses of variance, however the size of the current study population does not support this form of statistical analysis and is a limitation of the current study. The non-significant findings should not be equivocally interpreted as there being no difference at all. Given the small sample size, the study may have been underpowered to find a true significant difference. Another potential limitation of the present study is the issue of recall bias, in that subjects may have incorrectly reported a history of concussion.


The current study presents reference ranges for parameters associated with five KINARM robotic tasks that assess reaching, position sense, bimanual motor function, visuospatial skills, attention, and decision-making. When children/adolescents with and without a history of concussion are compared, no differences in parameter performance were detected.



Analysis of variance


Arm position matching




Multivariate analysis of variance


Object hit


Object hit and avoid




Trail Making B


Visually guided reaching


  1. 1.

    Hockey Canada. Hockey Canada annual report: 2013–2014. Available at: Accessed 23 Sept 2016.

  2. 2.

    Johnson LSM. Concussion in youth ice hockey: Its time to break the cycle. CMAJ. 2011;183(8):921–4.

  3. 3.

    Emery C, Meeuwisse WH, McAllister JR. Survey of sport participation and sport injury in Calgary and area high schools. Clin J Sport Med. 2006;16:20–6.

    Article  PubMed  Google Scholar 

  4. 4.

    McCrory P, Meeuwisse WH, Aubry M, Cantu B, Dvorak J, Echemendia RJ, Engebretsen L, Johnston K, Kutcher JS, Raftery M, Sills A, et al. Consensus statement on concussion in sport: The 4th international conference on concussion in sport held in Zurich, November 2012. J Am Coll Surg. 2013;216(5):e55–71.

    Article  PubMed  Google Scholar 

  5. 5.

    Frommer LJ, Gurka KK, Cross KM, Ingersoll CD, Comstock RD, Saliba SA. Sex differences in concussion symptoms of high school athletes. J Athl Train. 2011;46:76–84.

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Makdissi M, Darby D, Maruff P, Ugoni A, Brukner P, McCrory PR. Natural history of concussion in sport: markers of severity and implications for management. Am J Sports Med. 2010;38:464–71.

    Article  PubMed  Google Scholar 

  7. 7.

    Ewing R, McCarthy D, Gronwall D, Wrightson P. Persisting effects of minor head injury observable during hypoxic stress. J Clinic Neuropsy. 1980;2(2):147–55.

    Article  Google Scholar 

  8. 8.

    Leininger BE, Gramling SE, Farrell AD, Kreutzer JS, Peck EA. Neuropsychological deficits in symptomatic minor head injury patients after concussion and mild concussion. J Neur Psychiatry. 1990;53:293–6.

    CAS  Article  Google Scholar 

  9. 9.

    Alexancer PM. Mild traumatic brain injury: pathophysiology, natural history, and clinical management. Neurology. 1995;45:1253–60.

    Article  Google Scholar 

  10. 10.

    Yarnell P, Rossie GV. Minor whiplash head injury with major debilitation. Brain Inj. 1988;2(3):255–8.

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    Malojcic B, Mubrin Z, Coric B, Susnic M, Spilich GJ. Consequences of mild traumatic brain injury on information processing assessed with attention and short-term memory tasks. J Neurotrauma. 2008;25:30–7.

    Article  PubMed  Google Scholar 

  12. 12.

    McCrea M, Guskiewicz KM, Marshall SW, Barr W, Randolph C, Cantu RC, Onate JA, Yang J, Kelly JP. Acute effects and recovery time following concussion in collegiate football players. JAMA. 2003;290(19):2556–63.

    CAS  Article  PubMed  Google Scholar 

  13. 13.

    Halterman CI, Langan J, Drew A, Rodriguez E, Osternig LR, Chou LS, van Donkelaar P. Tracking the recovery of visuospatial attention deficits in mild traumatic brain injury. Brain. 2006;129:747–53.

    Article  PubMed  Google Scholar 

  14. 14.

    Mayr U, LaRoux C, Rolheiser T, Osternig, Chou LS, van Donkelaar P. Executive dysfunction assessed with a task-switching task following concussion. PLoS One. 2014;9(3):e91379. doi:10.1371/journal.pone.0091379.

    Article  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Subbian V, Ratcliff JJ, Korfhagen JJ, Hart KW, Meunier JM, Shaw GJ, Lindsell CJ, Beyette FR. A novel tool for evaluation of mild traumatic brain injury patients in the emergency department: Does robotic assessment of neuromotor performance following injury predict the presence of postconcussion symptoms at follow-up? Academic Emergency Med. 2016. doi:10.1111/acem.12906.

    Google Scholar 

  16. 16.

    Field M, Collins MW, Lovell MR, Maroon J. Does age play a role in recovery from sports-related concussion? A comparison of high school and collegiate athletes. J Pediatr. 2003;142:546–53.

    Article  PubMed  Google Scholar 

  17. 17.

    Moser RS, Schatz P, Jordan BD. Prolonged effects of concussion in high school athletes. Neurosurg. 2005;57(2):300–6.

    Article  Google Scholar 

  18. 18.

    Nelson LD, Guskiewicz KM, Barr WB, Hammeke TA, Randolph C, Ahn KW, Wang Y, McCrea MA. Age differences in recovery after sport-related concussion: a comparison of high school and collegiate athletes. J Athl Train. 2016;51(2):142–52.

    Article  PubMed  Google Scholar 

  19. 19.

    Emery CA, Tyreman H. Sport participation, sport injury, risk factors and sport safety practices in Calgary and area junior high schools. Paediatr Child Health. 2009;14:439–44.

    PubMed  PubMed Central  Google Scholar 

  20. 20.

    Marchie A, Cusimano MD. Bodychecking and concussions in ice hockey: should our youth pay the price? CMAJ. 2003;169:124–8.

    PubMed  PubMed Central  Google Scholar 

  21. 21.

    Emery CA, Kang J, Shrier I, Goulet C, Hagel BE, Benson BW, Nettel-Aguirre A, McAllister JR, Hamilton GM, Meeuwisse WH. Risk of injury associated with body checking among youth ice hockey players. JAMA. 2010;303(22):2265–72.

    CAS  Article  PubMed  Google Scholar 

  22. 22.

    Emery C, Kang J, Shrier I, Goulet C, Hagel BH, Benson B, Bettel-Aguirre A, McAllister J, Meeuwisse W. The risk of injury associated with body checking experience in youth ice hockey players. JAMA. 2011;183(11):1249–56.

    Google Scholar 

  23. 23.

    Black A, Macpherson AK, Hagel BE, Romiti MA, Palacios-Derflingher L, Kang J, Meeuwisse WH, Emery CA. Policy change eliminating body checking in non-elite ice hockey leads to a threefold reduction in injury and concussion risk in 11–12 year-old players. Br J Sports Med. 2016;50:55–61.

    Article  PubMed  Google Scholar 

  24. 24.

    Stewart TC, Gilliland J, Fraser DD. An epidemiologic profile of pediatric concussions: Identifying urban and rural differences. J Trauma Acute Care Surg. 2014;75(3):736–42.

    Article  Google Scholar 

  25. 25.

    McClincy MP, Lovell MR, Pardini J, Collins MW, Spore MK. Recovery from sports concussion in high school and collegiate athletes. Brain Inj. 2006;20(1):33–9.

    Article  PubMed  Google Scholar 

  26. 26.

    McLeod TCV, Bay RC, Lam KC, Chhabra A. Representative baseline values on the sport concussion assessment tool 2 (SCAT2) in adolescent athletes vary by gender, grade, and concussion history. Am J Sports Med. 2012;40:927–33.

    Article  Google Scholar 

  27. 27.

    Mayfield R, Bay RC, McLeod TCV. Post-concussion deficits measured by the sport concussion assessment tool 2 among interscholastic athletes. Athletic Training in Sports Health Care. 2013;5(6):265–71.

    Article  Google Scholar 

  28. 28.

    DeBeaumont L, Brisson B, Lassonde M, Jolicoeur P. Long-term electrophysiological changes in athletes with a history of multiple concussions. Brain Inj. 2007;21(6):631–44.

    Article  Google Scholar 

  29. 29.

    Fait P, Swaine B, Cantin JF, Leblond J, McFadyen BJ. Altered integrated locomotor and cognitive function in elite athletes 30 days postconcussion: a preliminary study. J Head Trauma Rehabilitation. 2013;28(4):293–301.

    Article  Google Scholar 

  30. 30.

    Howell DR, Osternig LR, Chou LS. Dual-task effect on gait balance control in adolescents with concussion. Arch Phys Med Rehabilitation. 2013;94:1513–20.

    Article  Google Scholar 

  31. 31.

    Howell DR, Osternig LR, Koester MC, Chou LS. The effect of cognitive task complexity on gait stability in adolescents following concussion. Exp Brain Res. 2014;232:1773–82.

    Article  PubMed  Google Scholar 

  32. 32.

    van der Naalt J, Hew JM, van Zomeren AH, Sluiter WJ, Minderhoud JM. Computed tomography and magnetic resonance imaging in mild to moderate head injury: early and late imaging related to outcome. Ann Neurol. 1999;46:70–8.

    Article  PubMed  Google Scholar 

  33. 33.

    Waljas M, Iverson GL, Lange RT, Hakulinen U, Dastidar P, Huhtala H, Liimatainen S, Hartikainen K, Ohman J. A prospective biopsychosocial study of the persistent post-concussion symptoms following mild traumatic brain injury. J Neurotrauma. 2015;32:1–14.

    Article  Google Scholar 

  34. 34.

    Davis GA, Purcell LK. The evaluation and management of acute concussion differs in young children. Br J Sports Med. 2014;48(2):98–101.

    Article  PubMed  Google Scholar 

  35. 35.

    Fuelscher I, Williams J, Hyde C. Developmental improvements in reaching correction efficiency are associated with an increased ability to represent action mentally. J Exp Child Psychol. 2015;140:74–91.

    Article  PubMed  Google Scholar 

  36. 36.

    Halberda J, Ly R, Wilmer JB, Naiman DQ, Germine L. Number sense across the lifespan as revealed by a massive internet-based sample. Proc Natl Acad Sci U S A. 2012;109(28):11116–20.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  37. 37.

    Valovich McLeod TC, Perrin DH, Guskiewicz KM, Shultz SJ, Diamond R, Gansneder BM. Serial administration of clinical concussion assessments and learning effects in healthy young athletes. Clin J Sport Med. 2004;14(5):287–95.

    Article  PubMed  Google Scholar 

  38. 38.

    Valovich McLeod TC, Barr WB, McCrea M, Guskiewicz KM. Psychometric and measurement properties of concussion assessment tools in youth sports. J Athl Train. 2006;41(4):399–408.

    PubMed  PubMed Central  Google Scholar 

  39. 39.

    Scott SH, Dukelow SP. Potential of robots as next-generation technology for clinical assessment of neurological disorders and upper-limb therapy. J Rehab Res Dev. 2011;48(4):335–54.

    Article  Google Scholar 

  40. 40.

    Dukelow SP, Herter TM, Moore KD, Demers MJ, Glasgow JI, Bagg SD, Norman KE, Scott SH. Quantitative assessment of limb position sense following stroke. Neurorehab Neural Repair. 2010;24:178–87.

    Article  Google Scholar 

  41. 41.

    Otaka E, Otaka Y, Kasuga S, Nishimoto A, Yamazaki K, Kawakami M, Ushiba J, Liu M. Clinical usefulness and validity of robotic measures of reaching movement in hemiparetic stroke patients. J Neuroeng Rehabil. 2015;12(66):1–10. doi:10.1186/s12984-015-0059-8.

    Google Scholar 

  42. 42.

    Massie CL, Du Y, Conroy SS, Krebs HI, Wittenberg GF, Bever CT, Whitall J. A clinically relevant method of analyzing continuous change in robotic upper extremity chronic stroke rehabilitation. Neurorehabil Neural Repair. 2015; 14: 1–10. DOI: 10.1177/1545968315620301

  43. 43.

    Basteris A, Jijenhuis SM, Stienen AHA, Buurke JH, Prange GB, Amirabdollahian F. Training modalities in robot-mediated upper limb rehabilitation in stroke: a framework for classification based on a systematic review. J Neuroeng Rehabil. 2014;11(111):1–15.

    Google Scholar 

  44. 44.

    Cho KH, Song WK. Robot-assisted reach training for improving upper extremity function of chronic stroke. Tohoku J Exp Med. 2015;237:149–55. doi:10.1620/tjem.237.149.

    Article  PubMed  Google Scholar 

  45. 45.

    Frisoli A, Procopio C, Chisari C, Creatini I, Bonfiglio L, Bergamasco M, Rossi B, Carboncini MC. Positive effects of robotic exoskeleton training of upper limb reaching movements after stroke. J Neuroeng Rehabil. 2012;9(36):1–16.

    Google Scholar 

  46. 46.

    Dukelow SP, Herter TM, Bagg SD, Scott SH. The independence of deficits in position sense and visually guided reaching following stroke. J Neuroeng Rehabil. 2012;9(72):1–13.

    Google Scholar 

  47. 47.

    Debert CT, Herter TM, Scott SH, Dukelow S. Robotic assessment of sensorimotor deficits after traumatic brain injury. J Neurol Phy Ther. 2012;36:58–67.

    Article  Google Scholar 

  48. 48.

    Coderre AM, Zeid AA, Dukelow SP, Demmer MJ, Moore KD, Demers MJ, Bretzke H, Herter TM, Glasgow JI, Norman KE, Bagg SD, Scott SH. Assessment of upper-limb sensorimotor function of subacute stroke patients using visually guided reaching. Neurorehab Neural Repair. 2010;24(6):528–41.

    Article  Google Scholar 

  49. 49.

    Tyryshkin K, Coderre AM, Glasgow JI, Herter TM, Bagg SD, Dukelow SP, Scott SH. A robotic object hitting task to quantify sensorimotor impairments in participants with stroke. J NeuroEngineering and Rehabil. 2014;11(47):1–12.

    Google Scholar 

  50. 50.

    Semrau JA, Herter TM, Scott SH, Dukelow SP. Robotic identification of kinesthetic deficits after stroke. Stroke. 2013;44:3414–21.

    Article  PubMed  Google Scholar 

  51. 51.

    Semrau JA, Herter TM, Scott SH, Dukelow SP. Examining differences in patterns of sensory and motor recovery after stroke with robotics. Stroke. 2015;46:3459–69. doi:10.1161/STROKEAHA.115.010750.

    Article  PubMed  Google Scholar 

  52. 52.

    Subbian V, Meunier JM, Korfhagen JJ, Ratcliff JJ, Shaw GJ, Beyette FR. Quantitative assessment of post-concussion syndrome following mild traumatic brain injury using robotic technology. Conf Proc IEEE Eng Med Biol Soc. 2014: 5353–6. Doi:10.1109/EMBC.2014.6944835

  53. 53.

    Benson B, Semrau J, Debert C, Scott S, Meeuwisse W, Dukelow S. Quantitative assessment of sensorimotor dysfunction and recovery using robotics in athletes sustaining an acute sport-related concussion. Br J Sports Med. 2013;47, e1. doi:10.1136/bjsports-2012-092101.32.

    Article  Google Scholar 

  54. 54.

    Tator CH, Davis H. The postconcussion syndrome in sports and recreation: clinical features and demography in 138 athletes. J Neurosurgery. 2014;75(Suppl4):S106–12.

    Article  Google Scholar 

  55. 55.

    Tator CH. Concussions and their consequences: current diagnosis, management and prevention. Canadian Medical Association J. 2013;185(11):975–9.

    Article  Google Scholar 

  56. 56.

    Little CE, Emery C, Black A, Scott SH, Meeuwisse W, Nettel-Aquirre A, Benson B, Dukelow S. Test retest reliability of KINARM robot sensorimotor and cognitive assessment: in pediatric ice hockey players. J Neuroeng Rehabil. 2015;12:1–18. doi:10.1186/s12984-015-0070-0. Accessed 23 Sept 2016.

  57. 57.

    Harmon KG, Drezner JA, Gammons M, Guskiewicz KM, Halstead M, Herring SA, Kutcher JS, Pana A, Putukian M, Roberts WO. American Medical Society for Sports Medicine position statement: concussion in sport. Br J Sports Med. 2013;47:15–26.

    Article  PubMed  Google Scholar 

  58. 58.

    Arbuthnott K, Frank J. Trail making test, Part B as a measure of executive control: validation using a set-switching paradigm. J Clin Exp Neuropsychol. 2000;22(4):518–28.

    CAS  Article  PubMed  Google Scholar 

  59. 59.

    Field A. Discovering statistics using SPSS. 3rd ed. London: SAGE Publications; 2009.

    Google Scholar 

  60. 60.

    Andrews S, Ellis DA, Shaw H, Piwek L. Beyond self-report: tools to compare estimated and real-world smartphone use. PLoS One. 2015. doi:10.1371/journal.pone.0139004.

    Google Scholar 

  61. 61.

    Stern JE, McLain AC, Buck Louis GM, Luke B, Yeung EH. Accuracy of self-reported survey data on assisted reproductive technology treatment parameters and reproductive history. Am J Obstet Gynecol. 2016. doi:10.1016/j.ajog.2016.02.010.

    PubMed  Google Scholar 

  62. 62.

    Neal SE, Hosegood V. How reliable are reports of early adolescent reproductive and sexual health events in demographic and health surveys? Int Perspect Sex Reprod Health. 2015;41(4):210–7.

    Article  PubMed  Google Scholar 

  63. 63.

    Almeida LR, Valenca GT, Negreiros NN, Pinto EB, Oliveira-Filho J. Comparison of self-report and performance-based balance measures for predicting recurrent falls in people with Parkinson Disease: cohort study. Phys Ther. 2016. doi:10.2522/ptj.20150168.

    Google Scholar 

  64. 64.

    Dey AK, Alvass A, Muir RT, Black SE, Swartz RH, Murray BJ, Boulos MI. Validity of self-report of cardiovascular risk factors in a population at high risk for stroke. J Stroke Cerebrovasc Dis. 2015;24(12):2860–5.

    Article  PubMed  Google Scholar 

  65. 65.

    Portney LG, Watkins MP. Foundations of Clinical Research: Application to Practice. 3rd ed. Upper Saddle River: Prentice Hall; 2009.

    Google Scholar 

  66. 66.

    Meehan WP, Mannix RC, O’Brien MJ, Collins MW. The prevalence of undiagnosed concussions in athletes. Clin J Sport Med. 2013;23(5):339–42.

    Article  PubMed  PubMed Central  Google Scholar 

  67. 67.

    Battikhi MN. Age-specific reference ranges for prostate-specific antigen (PSA) in Jordanian patients. Prostate Cancer Prostatic Dis. 2003;6(3):256–60.

    CAS  Article  PubMed  Google Scholar 

  68. 68.

    Krol AL, Mrazik M, Naidu D, Brooks BL, Iverson GL. Assessment of symptoms in a concussion management programme: method influences outcome. Brain Inj. 2011;25(13–14):1300–5.

    Article  PubMed  Google Scholar 

  69. 69.

    Jinguiji TM, Bompadre V, Harmon KG, Satchell EK, Gilbert K, Wild J, Eary JF. Sport Concussion Assessment Tool-2: baseline values for high school athletes. Br J Sports Med. 2012;46(5):365–70. doi:10.1136/bjsports-2011-090526.

    Article  Google Scholar 

  70. 70.

    Lovell MR, Iverson GL, Collins MW, Podell K, Johnston KM, Pardini D, Pardini J, Maroon JC. Measurement of symptoms following sports-related concussion: reliability and normative data for the post-concussion scale. Appl Neuropsychol. 2006;13:166–74.

    Article  PubMed  Google Scholar 

  71. 71.

    Iverson GL, Lange RT. Examination of “postconcussionlike” symptoms in a healthy sample. Appl Neuropsychol. 2003;10:137–44.

    Article  PubMed  Google Scholar 

  72. 72.

    Gasquoine PG. Postconcussional symptoms in chronic back pain. Appl Neuropsychol. 2000;7:83–9.

    CAS  Article  PubMed  Google Scholar 

  73. 73.

    Iverson GL, McCracken LM. ‘Postconcussive’ symptoms in persons with chronic pain. Brain Inj. 1997;11:783–90.

    CAS  Article  PubMed  Google Scholar 

  74. 74.

    Covassin TPD, Schatz PPD, Swanik CBPD. Sex differences in neuropsychological function and post-concussion symptoms of concussed collegiate athletes. Neurosurgery. 2007;61:345–51.

    Article  PubMed  Google Scholar 

  75. 75.

    Lovell MR, Solomon GS. Neurocognitive test performance and symptom reporting in cheerleaders with concussions. J Pediatr. 2013;163(4):1192–5.

    Article  PubMed  Google Scholar 

Download references


This research would not have been possible without the support of Hockey Calgary, players and parents.


This work was supported by a Canadian Institutes Health Research Team Grant (201210): Mild Traumatic Brain Injury in Children and Youth, Hotchkiss Brain Institute (Co-PI’s – CE, WM) and the Talisman Energy Fund in support of Healthy Living and Injury Prevention and the Alberta Children’s Hospital Research Institute for Child and Maternal Health. The funding body played no role in the design of the study and collection, analysis, and interpretation of the data or in writing the manuscript.

Availability of data and materials

Data will not be shared. This data is currently being used in other analyses.

Authors’ contributions

CEL performed the data analysis, statistical analysis, and drafted the manuscript. CE, SPD, SHS, and WM provided input to the study design. CEL, SPD, and CE, SHS contributed to the data analysis. CE, SPD, WM, LPD provided input on statistical analysis of the data. CE, SPD, WM, LPD and SHS were involved in drafting the manuscript. All authors read and approved the final manuscript.

Competing interests

SHS is co-founder and chief scientific officer of BKIN Technologies, the company that commercializes the KINARM robotic device. The other authors CEL, CE, WM, LPD, SPD declare that they have no competing interests.

Consent for publication

Parental consent was provided for the photograph in Fig. 1 to be included in the current publication.

Ethics approval and consent to participate

The Conjoint Health Research Ethics Board at the University of Calgary approved the study (Ethics ID number E24026).

Author information



Corresponding author

Correspondence to C. Elaine Little.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Little, C.E., Emery, C., Scott, S.H. et al. Do children and adolescent ice hockey players with and without a history of concussion differ in robotic testing of sensory, motor and cognitive function?. J NeuroEngineering Rehabil 13, 89 (2016).

Download citation


  • Robot
  • Sensorimotor
  • Cognitive assessment
  • Child/adolescent
  • Concussion
  • Ice hockey