A biologically inspired neural network controller for ballistic arm movements
© Bernabucci et al; licensee BioMed Central Ltd. 2007
Received: 22 May 2006
Accepted: 03 September 2007
Published: 03 September 2007
Skip to main content
© Bernabucci et al; licensee BioMed Central Ltd. 2007
Received: 22 May 2006
Accepted: 03 September 2007
Published: 03 September 2007
In humans, the implementation of multijoint tasks of the arm implies a highly complex integration of sensory information, sensorimotor transformations and motor planning. Computational models can be profitably used to better understand the mechanisms sub-serving motor control, thus providing useful perspectives and investigating different control hypotheses. To this purpose, the use of Artificial Neural Networks has been proposed to represent and interpret the movement of upper limb. In this paper, a neural network approach to the modelling of the motor control of a human arm during planar ballistic movements is presented.
The developed system is composed of three main computational blocks: 1) a parallel distributed learning scheme that aims at simulating the internal inverse model in the trajectory formation process; 2) a pulse generator, which is responsible for the creation of muscular synergies; and 3) a limb model based on two joints (two degrees of freedom) and six muscle-like actuators, that can accommodate for the biomechanical parameters of the arm. The learning paradigm of the neural controller is based on a pure exploration of the working space with no feedback signal. Kinematics provided by the system have been compared with those obtained in literature from experimental data of humans.
The model reproduces kinematics of arm movements, with bell-shaped wrist velocity profiles and approximately straight trajectories, and gives rise to the generation of synergies for the execution of movements. The model allows achieving amplitude and direction errors of respectively 0.52 cm and 0.2 radians.
Curvature values are similar to those encountered in experimental measures with humans.
The neural controller also manages environmental modifications such as the insertion of different force fields acting on the end-effector.
The proposed system has been shown to properly simulate the development of internal models and to control the generation and execution of ballistic planar arm movements. Since the neural controller learns to manage movements on the basis of kinematic information and arm characteristics, it could in perspective command a neuroprosthesis instead of a biomechanical model of a human upper limb, and it could thus give rise to novel rehabilitation techniques.
Human beings are able to accomplish extremely complex motor tasks in all kinds of environments by means of a highly organized architecture including sensors, processing units and actuators. From a cognitive and developmental perspective, and a rehabilitation standpoint, it is necessary to fully understand the complex interactions between the controller (the Central Nervous System) and the controlled object (all parts of the body). These interactions describe the process of motor control for which many theories have been developed. As far as the generation of motor commands is concerned, in literature it is generally acknowledged that nervous system generates motor commands based on internal models able to take account of the kinematics and the dynamics of the biomechanical structures [2–4]. These models can be described as groups of neural connections that intrinsically contain information about biomechanical properties of the human body in relation both to the environment and the subject's experience.
However, the mechanisms underlying the generation and organization of these neural models are still object of controversy . In order to interpret their functions, in literature different computational approaches to simulate both the biomechanical structure and the controller have been presented for a 2D  framework.
In this context, there is an interest in the use of Artificial Neural Networks (ANN) because of their capabilities to adapt and to generalise to new situations. In order to link the neural learning/adaptation processes to their artificial replica, ANN have been used in some studies regarding neurophysiologic simulations. However, most of these ANN imply the presence of a supervisor that uses sensory information in order to minimize the error related to the motor task . This methodology, commonly implemented on forward multilayer networks with retrospective learning (back propagation), is efficient from an operative standpoint, but not completely plausible as a biologically inspired learning model of motor control, at least for the presence of a teacher who is pre-existent to the organization of the system.
To overcome this drawback, neural models using unsupervised training techniques for the exploration of motor spaces have been proposed  thus meeting the features of self-organization typical of internal representations. The adaptability of the neural model together with the unsupervised training can also answer to environmental modifications such as those represented by external force fields and haptic distortions. Following this approach, it is of interest to study models able to simulate motor control mechanisms in terms of both generating and managing the sequence of motor commands that enable the arm to execute movements in the space. In this paper the focus is on the execution of ballistic movements.
According to the work of Karniel and Inbar , ballistic movements can be studied considering that: 1) there is no visual information; 2) any single movement is ballistic. As for every voluntary movement, the central nervous system must address three main computational problems: 1) determination of the desired trajectory in the visual coordinates; 2) transformation of the trajectory from visual to body coordinates; 3) generation of motor commands . The lack of visual information and the ballistic nature prevent to have a feedback on the controller [11, 12]: in fact, the delay introduced by a proprioceptive feedback in a biological system is too large to permit on-line corrections of the trajectories, and other studies  state that motor commands could be adjusted online without the need to involve a conscious decision process.
In any case, the commonly accepted idea is that ballistic movements can be managed by feed-forward controllers without using visual information as feedback. Some common characteristics are generally shared by ballistic movements on a plane, and these are: roughly straight pathways and bell-shaped hand speed profiles [14, 15]. Moreover, point to point movements have been studied following the hypothesis known as the minimum variance rule, able to attain physiological kinematic results as Fitt's Law and 2/3 Power Law . Some authors [17, 18] tried to provide a mathematical explanation of these kinematic invariants suggesting the hypothesis that the central nervous system aims at maximizing the smoothness of the movement.
In this work, ballistic movements will be controlled by an ANN controller that can be defined as "biologically inspired". It will be able to generate muscular activations knowing only the starting and arrival points of each movement, giving rise to a solution for the inverse dynamics problem (that is determining muscular forces on the basis of kinematic information). The muscular activations will generate ballistic movements having characteristics similar to human movements. This biologically inspired model will integrate an ANN, which should accomplish the task on the basis of its adaptability and plasticity [19, 20], together with a biomechanical arm model, considered as a 2 DOF system, in order to simulate the behaviour of an end-effector driven by the sequences generated by the controller.
In the first part of this work, materials and methods will be reported: after a description of the parallel distributed computational system that has been used, the generator of the neural input commands and the biomechanical model of the arm will be presented. Finally, the evaluation tests and the obtained results will be discussed.
In this section we describe the general scheme of the proposed model, which can be divided into three main modules, each one with a specific functionality in the transformation process from perception to motor action, that is: the perception task, the elaboration of data and the motor activation. Therefore, two computational blocks simulate the motor control of the upper limb, while a third block is responsible for the modelling of the actuator.
The first module is devoted to processing spatial information in order to solve the inverse dynamics problem (i.e. which neural signals, that is which forces, have to be generated to reach a specific point in the environment?). The strategy can be acquired after a series of synaptic modifications that represent the construction of the internal model both in architectural and functional ways. The whole process, that simulates the generation of the internal models by means of synaptic modifications, is called learning. It must be emphasized that, since the main purpose of the present work is to characterize a model simulating the generation and the actuation of ballistic movements, no online feedback on the position error is present in the scheme. We deal, in fact, with a process where the learning scheme modifies the neural features in order to map the working space and reach the desired targets. Even if the learning scheme can be considered as a functionality of the Neural System, a separate paragraph in the Materials and Methods section has been devoted to the explanation of the learning process in order to outline the processing scheme adopted.
The second module is called Pulse Generator, and it essentially generates the motor signals necessary for to activate the muscles and to consequently produce the movements of the arm model.
The third module simulates a simplified version of the biomechanical arm model. In fact, the human arm presents a high number of degrees of freedom and a redundancy due to the difference of dimensions between muscular activations space and working space (that is the whole set of the points attainable by the arm model), so that the set of available ways to accomplish a specific task is not unique. In the model, only two mono-articular pairs of muscles for each joint (elbow and shoulder) and a bi-articular pair of muscles connecting the two joints have been taken into account. The first agonist-antagonist pair acts across the shoulder joint: the pectoralis major is the flexor, while the deltoid is the extensor. The second pair acts across the elbow joint: the long head biceps brachialis is the flexor, while the lateral head triceps brachialis is the extensor. The third pair of muscles links both the joints: the flexor is the biceps brachialis short head and the extensor is the triceps brachialis long head.
From the results that will be presented below, it emerges that, even in this simplified version, the synthesized system is able to execute accurate planar movements.
The first module has been structured as a Multi Layer Perceptron with an architecture composed by 4 layers. The design process of the neural network used for this study is based on the analysis of the behaviour of various neural structures in responding to a same training and testing set.
In order to choose the most adequate structure, different types of neural networks have been considered and trained: a first group with only one hidden layer (varying the number of neurons), and a second group with two hidden layers (varying the number of neurons in different combinations for each layer). Experimental results considering the errors with respect to the training set and to the testing set as cross-validation (in order to avoid over-fitting problems) led us to choose an ANN design with two hidden layer of 20 neurons each.
The input layer is therefore defined by 4 input units, which correspond to the coordinates of the starting and final positions of the movement.
• for the shoulder, when the agonist muscle is activated, the movement starts. After a time interval, defined by the ANN, the antagonist is activated, so that the time interval TcoactShoulder is characterized by the co-activations of the agonist and antagonist (mono-articular) muscles of the shoulder joint (i.e. simultaneous presence of the neural inputs for shoulder muscles); its sign defines which muscle (i.e. agonist or antagonist) is activated first;
• for the elbow, TcoactElbow has the same function of TcoactShoulder;
• for the muscle pair that connect the two joints, TcoactBiarticular has the same function of TcoactShoulder and TcoactElbow.
• the movement duration is Tall: it represents the total duration of the neural activation, thus affecting the whole movement of the arm. This output value is constrained in the range 300 ms – 1 s. The time range has been chosen in order to let the limb model reach every sector of the working plane, while maintaining the ballistic characteristics of the movement.
Figure 2 depicts the profile of these neural activations having rectangular shapes, and shows the duration of the entire voluntary task ranging in the interval 300 ms and 1 s.
The transfer function chosen for every unit is the well known hyperbolic tangent , where the output nim of the ith neuron at the mth layer is obtained from the weighted outputs of the (m - 1)th level.
The values generated by the output layer, from now on indicated as neural outputs p, are bounded between -1 and 1, and are used by the Pulse Generator.
The system, in the present version, allows having only biphasic activation patterns for each muscle pair. Thus, the interval delimited by the initial point of the pattern and the TcoactShoulder, the TcoactElbow and the TcoactBiarticular values correspond to the Action Pulse, i.e. the time in which the neural activations of the agonist muscle determine an activation in the EMG signal, while the one going from this value till the end of the pattern, i.e. the time in which the neural co-activations of the antagonist muscle determine a braking burst in the EMG signal , corresponds to the Braking Command. The range of these intervals, including the co-activation time of the shoulder and the elbow muscles, together with the whole duration of the activations, establishes the direction, length and curvature of the movements.
The muscular system is thus based on 6 muscle-like actuators, and establishes the dynamic relationship between the position of the arm and the torques acting on each single joint.
Numerical values of the parameters of the arm
M – Mass of the subject
M1 – mass of the upper arm
M2 – mass of the lower arm
L – height of he subject
l1 – length of the upper arm
l2 – length of the lower arm
I1 – inertias of the upper arm
I2 – inertias of the lower arm
Numerical values of the Hill's parameters
where Φ = 0.6 and ϕ = 0.4 are non dimensional units and the F values in the equation are the values of the torque applied by each muscle of the corresponding joint during flexion or extension.
Finally, the trajectory in the working plane is obtained from a double integration at each sampling time of the acceleration of the end point of the effector due to the changes in the overall torque applied to both joints.
One key point of the present work is the training paradigm adopted for the neural controller with the aim of defining a specific internal model during ballistic movements of the arm, that is to establish a mapping between the desired movements within the working plane and the necessary neural outputs, so that the controller could learn the inverse dynamics of the biomechanical arm model. The algorithm will adapt the neural weights and biases so that, if the 4 inputs of the network respectively correspond to the coordinates of the starting point [q1, q2], and of the desired target [q1d, q2d], then the output of the net will approach the correct p.
The key feature of this approach is that the position error in executing the movements is not used in the training. The reason is that, following the studies of  a supervised training mechanism for the controller must be excluded, thus meaning that the knowledge of the position error made in carrying out the movement will not be used to train the neural network. The exclusion of a feedback circuit both in the phases of learning and executing the task, reflects the capacity of the motor control system to explore the workspace either without basing itself on pre-existent information (batch supervised training) or elaborating the data coming from the environment (feedback error learning). In the learning phase of the network, the association: "starting point – neural inputs generating the movement from the starting point to an ending point" is therefore used. This is the step-by-step procedure in which the controller learns to make different movements.
It is important to stress again that, unlike most of the models proposed in the literature, this controller learns the movement actually carried out, not the wanted one. This training strategy recalls the big picture of the classical Piagetian's concept of motor development. More in particular it can be considered as leading the way to the circular reaction learning model. Otherwise, in the proposed scheme, the construction of the inverse dynamics of the arm within a particular environment neglects the interconnection between the eye and the arm systems, but is driven by a purely proprioceptive exploration phase outlining the development of an internal model. During the training phase, the neural controller tends to achieve an optimal behaviour in reaching a desired target point by improving the correlation between the sensory map (starting and ending point) and the motor map (muscular activations which generate the movement between these two points) through the entire working plane. The reduction of the error on the final position can be thus considered as a consequence and not a cause of the learning procedure. The proposed neural model, basing on the philosophy architecture of Direct-Inverse Model (Jordan, 1995), shows novel and innovative characteristics.
During the training, the system automatically and randomly chooses the starting and ending points of the movements, which in turn determine the parameters p to be used in the Pulse Generator.
In addition, during the training a random noise generator acts on the output of the neural network in order to prevent convergence on local minima, which would imply a limitation in direction or amplitude of upper limb movements.
In the initial phases of the training, the controller is not trained, and there is no correspondence between the desired target and the one actually reached by the movement of the biomechanical model of the arm. At the end of each task, a standard back-propagation algorithm with momentum is used for the training and thus the variation of the weights.
The training of the artificial neural network and the complete coverage of the working plane, with respect to both the possible starting and target points, can be reached with about 200.000 random generations (epochs). The decision about the end of the training is not based on a prefixed number of movements/training steps but on the monitoring of the convergence of the network.
Once the neural controlled is trained, the overall system is tested and the behaviour is analyzed. In this second phase, the noise generator is not active. Even if the inputs driving the network are different from those used in the training phase, the generalization capabilities of the connectionist system enables it to operate correctly.
Tests, and comparisons with results available in literature have been performed in order to evaluate the performance of the model after the convergence of the network.
The neural controller has been tested by presenting a high number of pairs of randomly chosen start-target points, and the errors in reaching the target have been recorded. Initially, in order to qualitatively test the behaviour of the controller, a set of 1000 movements starting from the same initial point have been considered. This set have been used to analyze the capacity to cover the entire workspace, to give a graphical representation of the correlation of the error position with respect to the length of the movements and to observe the distribution of the peak velocity within the working plane.
Furthemore,1200 random movements ranging from 5 cm to 60 cm, subdivided into groups spaced out by 5 cm (200 movements per group), have been generated, in order to make a comparison with the kinematic analysis of ballistic arm movements presented in literature (such as in [8, 14, 24]), where movements with a maximum amplitude of ± 30 cm have been examined. This subset has been defined Physiological Subset (PS). The characteristics of these tasks have been analyzed and compared to the data obtained from experimental tests on human beings, carried out in [8, 24]. In the latter paper, indexes useful to quantitatively determine some characteristics of the movements have been calculated.
The accuracy of the neural network in implementing the movements has been characterised by means of the following parameters:
• The absolute position error of the arrival position reached by the end-effector with respect to the desired final position (or target).
• The module error (the amplitude error).
• The phase error (the error pointing at the target).
• The curvature.
• The velocity curve.
The position and phase errors have been chosen in order to reveal the presence of a biased behaviour. In particular, the module error |e| has been defined as the difference between the segment connecting the starting point and the arrival point (xa, ya) and the straight line from the starting point to the target (xt, yt).
The coefficient of variation (CV), defined as the ratio between the standard deviation and the mean error position has also been evaluated. The distribution of the neural activation times with respect to the length of the movements has been taken into account.
Finally, the performance of the model with respect its possibilities of adapting to modifications in the environment, such as the presence of disturbing force fields, has been taken into account. To this purpose, a force proportional to the movement speed and directed along the horizontal axis has been inserted in the model, after the training for unobstructed movements in all the working plane. The additional training necessary to the model to be able to cope with this force and the performance as for the reaching errors have been evaluated.
The proposed neural system is able to achieve a complete coverage of the working plane, unlike other models  which are limited to short amplitude motor tasks, usually around 20–30 cm.
The first movement of the set simulates the role of the Pectoralis Major, in the shoulder joint, for targets positioned in a position west with respect to the starting point, while the second one implies the use of the Deltoid for the target allocated in a position east with respect to the starting point. The velocity profile reflects the bell shaped behaviour typically found in literature (see e.g. ).
When analysing the CV of the movements in PS it is possible to observe that monotonically increases ranging from 0.6 to 0.8. This behaviour can be explained by considering that when the movement becomes longer the precision in reaching the target decreases and the position error distribution increases. A comparison between the experimental data reported in [24, 26] and the data extracted from the simulated model of the present work is interesting because it puts in evidence the behaviour of the proposed neural model for as what concerns the curvature.
Mean values of the curvature indexes for the set of movements
Normal Curvilinearity NC
Maximum Curvilinearity MxC
Medium Curvilinearity MdC
Total Curvilinearity TC
Two main things must be stressed out:
• even if the biomechanical arm model is only an approximation of a real upper limb structure, in which further muscle activations have an influence, even if minor, on the overall movement, the results are very interesting.
• all the experiments on human subjects from the literature are replications of the same set of movements in different direction or with different amplitude; this brings a specialization of the tasks during the trials and therefore to lower errors.
In [26, 29] the normalized maximum curvature shows a value of about 0.05 ± 0.02. This result has been estimated as the ratio between the maximum distance from the straight line connecting the starting and the arrival point (that is the value MxC of the present system) and the length of straight line connecting them; moreover the values reported are related to tasks performed on the sagittal plane.
A neural-network motor controller able to simulate the ballistic movements of an arm has been presented. This controller is implemented by means of a neural network that simulates the internal model devoted to the management of the feed-forward aspects of the movement. The biomechanical model includes three pairs of muscles, and two joints.
The results obtained are plausible from a biological standpoint and might be interpreted taking into account some features:
• the capability of the controller to solve the inverse dynamics problem, that is to generate the proper muscular activations and then the muscular forces, exclusively on the basis of kinematic information such as the starting and ending point of the movements;
• the capacity of the neural controller to acquire the internal model of the plant with a learning process that excludes the use of an online feedback on the position error, thus showing a biologically plausible behaviour;
• the ability of the overall system to obtain realistic trajectories and bell shaped profiles similar to the experimental ones: the value of the parameters characterising the trajectories are in good agreement with those obtained from experiments on humans in similar tasks;
• the paradigm adopted for the on-line learning of the system dynamics that includes the biomechanical characteristics of the arm. In this way, both the adaptive characteristics of the controller with respect to the plant, and the simplicity of the control activations are emphasised.
Even if the model can be further complicated by optimizing the biomechanical model, to increase its capacity to obtain realistic trajectories, the presented results open a wide field of applications: from the cognitive ones to the use of this model for the control of smart Functional Electrical Stimulation (sFES) systems, to the rehabilitation. For instance, the availability of such a controller, once adapted to electrical stimulation systems, could enhance the possibilities of paretic patients to control their arm movements with more reduced effort during rehabilitation sessions, thus stimulating cortical synaptic plasticity and the recovery of correct muscular synergies.
The availability of such a controller, once adapted to electrical stimulation systems, can greatly enhance the possibilities of paretic patients to control their arm movements with more reduced effort during rehabilitation sessions, thus stimulating cortical synaptic plasticity and the recovery of correct muscular synergies. Such an application is quite new because most paretic patients have neural implants for grasping, while better and more physiological movements imply the involvement also of the entire arm. This controller not necessarily must rely on implanted cuff electrodes and could use surface and/or minimally invasive stimulators. The system can be driven by some HCI (for instance, a gaze tracker) which gives the neural controller the intention of the movement (starting and ending points) leaving the burden of activating the stimulation to the neural net. Obviously, the system proposed is able only to make ballistic planar movements, which in any case constitute as a proof of concept.
This work has been partially supported by MIUR.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.