Reaching within a dynamic virtual environment
Journal of NeuroEngineering and Rehabilitation volume 4, Article number: 23 (2007)
Planning and execution of reaching requires a series of computational processes that involve localization of both the target and initial arm position, and the translation of this spatial information into appropriate motor commands that bring the hand to the target. We have investigated the effects of shifting the visual field on visuomotor control using a virtual visual environment in order to determine how changes in visuo-spatial relations alter motor planning during a reach.
Five healthy subjects were seated in front of an immersive, stereo virtual scene while reaching for a visual target that remained stationary in space or unpredictably shifted to a second position (either to the right or left of the first target) with different inter-stimulus intervals. Motion of the scene either matched the motion of their head or was rotated counter clockwise at 130 deg/s in the roll plane.
Initial results suggested that both the temporal and spatial aspects of reaching were affected by a rolling visual field. Subjects were able to amend ongoing motion to match target position regardless of scene motion, but the presence of visual field motion produced significantly longer pauses during the reach movement when the target was shifted in space. In addition, terminal arm posture exhibited a drift in the direction opposite to the roll motion.
These findings suggest that roll motion of the visual field of view interfered with the ability to imultaneously process two consecutive stimuli. Observed changes in arm position following the termination of the reach suggest that subjects were compensating for a perceived change in their visual reference frame.
During the execution of a motor task, the central nervous system (CNS) monitors online body orientation by updating the internal representation of visual space. Studies have shown that both young and elderly healthy subjects are able to amend their ongoing movements in response to target displacement during a "double-step" paradigm which changes the spatial goal of the movement by unexpectedly changing the location of a visual target [1–4]. However, these movements have only been tested in stationary visual environments. During most active motions the individual and the external world are moving at the same time. While there is ample evidence that dynamic visual inputs affect motor behavior, (e.g., disrupting upper extremity movement trajectory and endpoint  and increasing postural instability [6, 7]), the weighting of such visual information and the exact role that visual motion plays in human motor control is not well understood. In recent years, virtual reality technology has emerged as a powerful tool to study motor control in healthy subjects and in patients with stroke or labyrinthine deficiency [8–10] because it enables us to manipulate the visual world. In the current study we examined how motion of a virtual environment (VE) might affect planning and execution of three-dimensional (3D) reaching movements using the double-step paradigm. We hypothesized that roll motion of the visual field, which was found to produce robust postural changes , would affect timing and position of the arm in space. Furthermore, we hypothesized that reaching toward a remembered target location would enhance this effect of visual field motion on performance.
Materials and Methods
Five young healthy adults (age 25–35 years) participated in the study. All subjects were right-handed and had normal or corrected-to-normal vision. Subjects gave informed consent in accordance with the Institutional Review Board of Northwestern University.
Apparatus and data collection
The VE and the hardware and software responsible for its generation have been previously reported . In brief, subjects were exposed to an immersive 3D wide field of view VE (scene), projected onto a 2.6 m × 3.2 m back-projection screen. Visual targets, which appeared with the scene, were generated as 3D virtual ball-shaped targets with a 1 cm radius (Figure 1a and 1b). Current orientation of the stereo shutter glasses, worn by the subject (Crystal Eyes, StereoGraphics Inc.), determined the correct perspective for the scene. Hand 3D movements were recorded using a six camera Motion Analysis system (Motion Analysis, Inc.). Reflective markers attached to the right arm, head, and trunk, were tracked at 120 Hz.
Subjects sat 1.2 m from the screen for two experimental protocols that controlled sequence and duration of the targets' appearance. In the first experiment, five blocks of trials were presented, each containing 12 single-step and 24 double-step trials in random order. The visual scene was either matched to motion of the head, or it rotated counter clockwise about the line of sight (rolling scene) at a constant velocity of 130 deg/s. In a single-step trial, a visual target appeared for 2 s. In a double-step trial, the central target appeared. Following a pre-specified inter-stimulus interval (ISI) of 50, 200 or 500 ms (either before or following movement initiation), the location of that target shifted either left or right and remained in the new position for 2 s. The scene either remained matched to head motion or started to roll as soon as a target appeared within the scene. Subjects were instructed to reach toward the target as soon as it appeared, and to move the hand towards the new location if the target changed position, (head and trunk were free to move). Subjects were also instructed to keep their hand at the final position until the scene turned black which signaled the end of the trial.
To investigate whether reaching toward a remembered target location enhanced the effect of roll motion on performance, a second experiment tested changes in the duration of target appearance. The target in the single-step condition and the final target in the double-step condition were visible for only 200 ms. ISI values were 200 and 500 ms. Each block contained a mixture of 12 single-step and 16 double-step trials.
Data from the measured hand position were low-pass filtered off-line at 8 Hz using a 4th order Butterworth digital filter. A 4% peak velocity threshold determined movement onset and offset. Hand path and the proportion and duration of pauses that occurred during the course of the movement (an interval of at least 40 ms in which the hand was stationary) were calculated from hand position and velocity.
Kinematics of the reaching motion within the VE was characterized by similar properties to those described in previous studies of reaching in the physical world, e.g., . Subjects were able to amend their ongoing motion in response to target displacement in both experiments and with both scene conditions. They exhibited single- and double-peaked bell-shaped velocity profiles for the single- and double-step conditions, respectively. In addition, some double-step movements exhibited a pause prior to modifying the arm trajectory (Figure 2a). The proportion of paused movements in the total double-step movements was lower for Experiment 1 (15.7%) than Experiment 2 (27.7%). In addition, mean duration of paused movement was significantly shorter in Experiment 1 than Experiment 2 (117 vs. 156 ms; χ2 (1) = 5.57, p = 0.018). Although the proportion of paused movements was similar with both scene conditions in both experiments, the duration of the pause was significantly different between the scene that was matched to head motion (120 ms) and the rolling scene (190 ms) in Experiment 2 (χ2 (1) = 15.8, p < 0.0001). Furthermore, the 3D hand path was consistently curved for both single- and double-step conditions, irrespective of target position and scene condition. Overshoots but especially undershoots with respect to the subject's body were observed across all subjects.
Finally, no obvious changes in the trunk and head position were observed during both experiments. In addition, all subjects were able to maintain the final hand position for both scene conditions in Experiment 1 (Figure 2b). Following the main reaching movement with a rolling scene in Experiment 2, however, the hand continuously moved slowly toward the right which was opposite the direction of the rolling scene (Figure 2c).
An earlier study  which used the double-step task in a stationary VE demonstrated that subjects modified hand trajectory in response to target displacement. We have now shown that this holds true with a moving virtual scene suggesting that our results will transfer to the physical world. Both the temporal and spatial aspects of the reach movement were affected by roll motion of the visual scene. Reach was affected both during and following the movement towards the target. Mean pause duration during the course of the reaching movement increased with roll of the visual scene, implying that visual motion interfered with the ability to simultaneously prepare motor responses to the two consecutive visual targets [4, 11]. Following termination of the reach, a drift in hand position was observed only during roll motion and in the absence of a target (Experiment 2). We infer that subjects were compensating for motion of the visual surround which produced a perceived change in their visual reference frame.
No postural changes were observed in our data even though a conflict existed between the signals from the visual, vestibular, and somatosensory systems . The absence of body tilt suggests that support surface inputs from the seat were more heavily weighted than the sensory conflict arising from roll motion of the visual field.
These initial results demonstrate that motion of the visual field affected both planning and execution of the reaching movement, particularly while reaching toward a remembered target. Reaching within a moving visual environment involves complex sensorimotor transformations as a result of the continuous change in the visual reference frame which could be used to promote adaptation and learning in individuals with CNS injury. Thus these data could eventually lead to rehabilitation paradigms that minimize the disturbing effect of visual motion on motor planning and execution. Knowledge of how visual motion affects both reaching and postural control might have ramifications for both elderly subjects and labyrinthine deficient individuals who have been shown to experience post-rotation disturbances of posture, gait, and arm movement control .
Farnè A, Roy AC, Paulignan Y, Rode G, Rossetti Y, Boisson D, Jeannerod M: Visuo-motor control of the ipsilateral hand: Evidence from right brain-damaged patients. Neuropsychologia 2003, 41: 739-757. 10.1016/S0028-3932(02)00177-X
Henis EA, Flash T: Mechanisms underlying the generation of averaged modified trajectories. Biol Cybern 1995, 72: 407-419. 10.1007/BF00201416
Martin O, Julian B, Boissieux L, Gascuel JD, Prablanc C: Evaluating online control of goal-directed arm movement while standing in virtual visual environment. J Visual Comput Animat 2003, 14: 253-260. 10.1002/vis.322
Dvorkin AY: Space representation using multiple reference frames in the motor system. PhD thesis. Hebrew University of Jerusalem, Israel; 2004.
Cohn JV, DiZio P, Lackner JR: Reaching during virtual rotation: Context specific compensations for expected coriolis forces. J Neurophysiol 2000, 83: 3230-3240.
Keshner EA, Kenyon RV, Langston J: Postural responses exhibit multisensory dependencies with discordant visual and support surface motion. J Vestib Res 2004, 14: 307-319.
Previc FH: The effects of dynamic visual stimulation on perception and motor control. J Vestib Res 1992, 2: 285-295.
Dvorkin AY, Shahar M, Weiss PL: Reaching within video-capture virtual reality: Using VR as a motor control paradigm. CyberPsychol Behav 2006, 9: 133-136. 10.1089/cpb.2006.9.133
Keshner EA, Kenyon RV: Using immersive technology for postural research and rehabilitation. Assist Technol 2004, 16: 54-62.
Viau A, Feldman AG, McFadyen BJ, Levin MF: Reaching in reality and virtual reality: a comparison of movement kinematics in healthy subjects and in adults with hemiparesis. J NeuroEng Rehab 2004, 1: 11. 10.1186/1743-0003-1-11
Plotnik M, Flash T, Inzelberg R, Schechtman E, Korczyn AD: Motor switching abilities in Parkinson's disease and old age: temporal aspects. J Neurol Neurosurg Psychiatry 1998, 65: 328-337. 10.1136/jnnp.65.3.328
Lackner JR, DiZio P: Vestibular, proprioceptive, and haptic contributions to spatial orientation. Annu Rev Psychol 2005, 56: 115-147. 10.1146/annurev.psych.55.090902.142023
This work was supported by NIH-NIDCD grant DC05235. We thank Jake Streepey and Leo Wu for their assistance and helpful discussions. We gratefully acknowledge VRCO for supplying CAVE library software. Consent was obtained from the individual in figure 1 to use his image.
The author(s) declare that they have no competing interests.
AYD designed and conducted the experiment, performed the analysis and wrote the manuscript. RVK participated in the design of the study and was involved in revising the manuscript. EAK participated in the design of the study, made substantial contribution for the interpretation of data and was involved in revising the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Dvorkin, A.Y., Kenyon, R.V. & Keshner, E.A. Reaching within a dynamic virtual environment. J NeuroEngineering Rehabil 4, 23 (2007). https://doi.org/10.1186/1743-0003-4-23
- Virtual Environment
- Visual Target
- Hand Position
- Roll Motion
- Target Displacement