Optimized intelligent control of a 2-degree of freedom robot for rehabilitation of lower limbs using neural network and genetic algorithm

Background There is an increasing trend in using robots for medical purposes. One specific area is rehabilitation. Rehabilitation is one of the non-drug treatments in community health which means the restoration of the abilities to maximize independence. It is a prolonged work and costly labor. On the other hand, by using the flexible and efficient robots in rehabilitation area, this process will be more useful for handicapped patients. Methods In this study, a rule-based intelligent control methodology is proposed to mimic the behavior of a healthy limb in a satisfactory way by a 2-DOF planar robot. Inverse kinematic of the planar robot will be solved by neural networks and control parameters will be optimized by genetic algorithm, as rehabilitation progress. Results The results of simulations are presented by defining a physiotherapy simple mode on desired trajectory. MATLAB/Simulink is used for simulations. The system is capable of learning the action of the physiotherapist for each patient and imitating this behaviour in the absence of a physiotherapist that can be called robotherapy. Conclusions In this study, a therapeutic exercise planar 2-DOF robot is designed and controlled for lower-limb rehabilitation. The robot manipulator is controlled by combination of hybrid and adaptive controls. Some safety factors and stability constraints are defined and obtained. The robot is stopped when the safety factors are not satisfied. Kinematics of robot is estimated by an MLP neural network and proper control parameters are achieved using GA optimization.


Background
The process of strengthening muscles to their normal values is a costly labor which requires time and patience [1]. This process is named rehabilitation. An intelligent instrument that replaces the duty of the physiotherapist and can accomplish such routine physical movements without the guidance and assistance of a physiotherapist will simplify the process and lower the costs drastically [2]. There are many exercise machines for rehabilitation purposes like CPMs [2]. Nevertheless, these machines are used only for ankle function and because of their low degree of freedom, their poor dynamic efficiency and prospect of high expense, they are used limitedly [3,4]. The most important machines used widely in many medical centers for therapy and rehabilitation purposes are LOKOMAT [5], ALEX [6] and LOPES [7]. These machines have high degree of freedom but their high cost causes them to be used limitedly. Moreover, their manipulation is hard and requires ingenuity. In addition, control system design is one of the major difficulties in construction of rehabilitation robots. Different approaches were developed to control movement of robot-aided therapy attached to human limbs [1,[8][9][10][11][12]. It is observed that the devices developed for rehabilitation purpose usually employ two control methods including hybrid control (force and position control) and impedance control. Intelligent techniques, which are optimized based on therapy session, were used in few works [2,9].
The main purpose of the developed system in this study is to introduce a low-cost system to satisfy the patient safety by a flexible structure controlled by an intelligent control strategy. The control parameters will be changed based on the therapy of different stages and patient qualification, thus the hybrid and adaptive control are used for controlling the suggested system. Neural networks are employed as the reference input of the proposed controller. Control parameters are optimized based on therapy sessions and safety factors and for this purpose a genetic evolutionary algorithm was applied. The suggested system can be used for rehabilitation of two limbs/joints (knee and hip).

Rehabilitation mode
The proffered structure is based on the flexion and extension movement for knee and hip [1]. For this purpose a 2-DOF planar robot is defined that can be attached to the trunk of lower limb ( Figure 1).
In Figure 1, link 1 is aligned to the thigh and link 2 is aligned to the shank and q 1 ٫q 2 are the angles of hip and knee, respectively and the limits of them are based on the flexion-extension of knee and hip process shown in Figure 1 and are written as: In robotic rehabilitation, the desired trajectory of manipulator obtained from the physiotherapist and then the related variables of the robot such as angles and velocity of them are computed based on inverse kinematic problem. Thereafter these parameters are used for control of robot to track the desired trajectory. These issues will be described in the next stages.

Control strategy used in proposed algorithm
As mentioned earlier, neural networks are employed as the reference input of proposed control algorithm. Thus, the neural network and its usages in proposed strategies are explained first and then the proffered control strategy will be described.

Neural network
An important area of neural networks application is in the field of robotics. Usually, these networks are designed for learning and reconstructing complex nonlinear mapping and have been widely used in the identification and control of a manipulator, which is the most important form of an assistant robot, and in tracking a trajectory based on sensory information. Generally, kinematics of parallel robot are non-linear problems and difficult to solve, thus an MLP neural network is used to estimate the joint variables. The second idea in using neural networks is originated from the results of experiments showing that there are training vulnerability centers in the adult mammalian spinal cord which activate and control motor neurons that are responsible for walking patterns [13][14][15]. These walking patterns that have been previously been reserved in can be replaced by other neurons. It means each neuron can be considered as a walking pattern.
The MLP neural network used in suggested method has two layers tansig activation function in layer (1) and activation purelin function in layer (2). The best number of neurons in layer (1) is obtained from an iteration algorithm. The Levenberg-Marquardt back propagation or trainlm algorithm is used for network training.

Control strategy
Control strategies of rehabilitation systems can be classified into three categories: force control, position control, position and force control [16,17]. Nevertheless, unlike industrial robots, rehabilitation-aided robots must be configured for stable, safe and compliant motion while interacting with humans [18]. The impedance control strategy propounded by Hogan [17,19] is one of the most appropriate approaches for such applications. Impedance control aims at controlling the position and force by adjusting the mechanical impedance of the manipulator to the external forces generated by contact with the manipulator's environment. Mechanical impedance is roughly an extended concept of the stiffness of a mechanism against a force applied to it [20].
The control block diagram used for rehabilitation purpose shown in Figure 2.
Therefore, the necessary joint torques to obtain desired impedance parameters are computed as: (Eq. (3) is obtained from the dynamic equation of robot manipulator that is in contact with its environments in joint space [2]): Where the y and subscript y denote the task space and the q denotes the joint space.
In this equation τ 2*1 is the torque input vector, q d 2*1 is the joint vector, y 2*1 is the manipulator's end effector vector, q 2*1 is the joint angle vector, h N (q٫˙q) 2*1 is the Coriolis and centrifugal force effects and other effects (such as gravity), M(q) 2*2 is the inertia matrix, M d 2*2 is the desired inertia matrix, J 2*2 is the Jacobean matrix, D d 2*2 is the desired damping coefficient matrix, K d 2*2 is the desired stiffness coefficient matrix and F 2*1 is external force exerted on the end-effector by its environment (this force can be defined as action and reaction force between patient and end-effector).
The term R 2 s Csþb ð Þ denotes the transfer function of any arm of robot, equipped with a DC servo motor, where R is the gear reduction ratio in motor and the parameters C and b are the effective moment of inertia and viscous friction coefficient, respectively [15]. 1 sþT is the amount of approximated delay.
In the applied control structure, Neural Network box is used to convert y d (desired position) to q d (desired joint angles). And y is the target of NN. In this block diagram, it is assumed that: Where γ e represents the error, or deflection of the MP (y) from its reference/desired position (y d ) and q e represents the error, or deflection of the joints (q) from its desired position (q d ).

Patient safety in the proposed algorithm
Patient safety is one of the most important factors in rehabilitation systems and can be guaranteed by the stability of software and hardware. Stability conditions for robotic systems under impedance or hybrid controllers had been investigated in some researches [1,8,17,19]. In this paper, a new asymptotic stability conditions for stiffness and impedance controllers is applied using an appropriate Routh approach [15] based on the relationship between a joint angle of the robot and desired trajectory. Corresponding transfer function can be defined as: According to Eq. (3) and the following substitutions: Where g is the gravitational acceleration, m is the mass of patient leg, L g is distance between the joint and the mass center of link and N is the effects (Coriolis and centrifugal force effects and other effects such as gravity) without static (coulomb) friction (friction ignored). The transfer function of Eq. (3) will be: The denominator polynomial is: After determining the stability conditions of controller gains based on Routh's theory [15], and taking into account that (M,K,D) are positive definite matrices, there will be: If we consider one of the joints of suggested robot (knee joint) and small movement these substitutions will be obtained: Now the transfer function of (5) will be: The denominator polynomial is: Then the stability condition will be: As shown in the next sections, the deviation of actual path from the desired path is considered as another system stability condition. In this paper, safety is guaranteed since some of the controller parameters can be adapted under the following criteria: 1-The stability constraints in (11) or (13) 2-Desired deviation or difference between actual and desired path (ΔP d will be explained in next section). 3-Different stroked patients (obtained from physiotherapist). 4-Different states of progression in the therapy process (by progress of rehabilitation steps and improvement in movement or feeling less pain obtained from physiotherapist). 5-The action/reaction force (F) between patient and robot (by a force sensor).
The robot is stopped when the safety factors is not satisfied. Thus, the recommended control strategy will be based on the combination of two strategies: impedance control and adaptive strategy. Controller parameters are finely tuned using a constrained non-linear optimization strategy such as GA that will be discussed in next section.

Optimization of control parameters
Strategies for solving old optimizing algorithm problems mostly depend on kind of aim, limit factors (linear, nonlinear) and types of applied variation in sampling (true and natural). This is a fact that old optimizing approaches, causes limitations in solving mathematical programming and applied research approaches and this is mainly because of intrinsic solving mechanism in those approaches. One of the main features of old optimizing algorithms is their inflexibility for the supposed problem and it's adaptation to possible and dynamic changes. Optimization means the way that tries to find the best parameter values in a function and in this paper the minimal deviation between actual and desired path must be found. For this purpose the classic strategies of optimal control can be used and by getting transfer function, the optimal parameters ((M d ٫K d ٫D d )٫F) are found to minimize the following cost function: Where e(t) is the deviation between actual and desired path and n is the number of stages in rehabilitation mode. Nevertheless, using these classic strategies result in more complexity of optimization problem and probably not finding a closed form answer owing to two reasons: firstly the feedback loop in block diagram is not identical for different cases (robotics kinematic are different). Secondly, because the parameters are in matrix form, an increase in their number, results in the increase of matrix dimensions. Therefore, defining an alternative strategy without the transfer function in order to minimize the cost function can be useful in decreasing complexity.
Eq. (15) is used for calculation of deviation between actual and desired path [20].
Where C is the compliance matrix and it is defined as: Where K is the stiffness matrix and J * is defined as: Now the impedance control parameters are modified so that cost function (18) can be minimized: In this case, because of the interaction between robot and human, the amplitude of force F is very important and its high value can damage the patient. Therefore, the cost function is rewritten as: The threshold of force is changed based on the therapy of different stages and patient qualification. We can incorporate constraint of F in the cost function (19) and define a new cost function as: Where α٫β٫h are changed based on the therapy of different stages and patient improvement (adaptive strategy). h can be called the accuracy factor as larger values of h will result in higher accuracy. Now the control parameters such as (M d ٫K d ٫D d ) and even F used for determination of necessary torques of links based on Eq. (3) are optimized by using a genetic evolutionary algorithm that will be explained in the next section.

Genetic algorithm
GA is a multi-purpose search and optimization algorithm that is inspired by the theory of genetics and natural selection. The problem to be solved using GA is encoded as a chromosome that consists of several genes. The solution of the problem is represented by a group of chromosomes referred to as a population. In each iteration of the algorithm, the chromosomes in the population will undergo one or more genetic operations such as crossover and mutation. The result of the genetic operations will become the next generations of the solution. This process continues until either the solution is found or a certain termination condition is met. The idea behind GA is to have the chromosomes in the population to slowly converge to an optimal solution. At the same time, the algorithm is supposed to maintain enough diversity so that it can scan a large search space. It is the combination of these two characteristics that makes GA a good search and optimization algorithm.
In the suggested algorithm value representation is used and the cost function is considered as Eq. (20). The main goal is to reach the minimum level of (ΔP) considering (F) which will not be higher than the defined threshold. On the other hand, since the parameters are multi-dimensional, chromosomes will be multidimensional instead of being a linear vector. In this case, M d ٫K d ٫D d and F will be the genes of each chromosome.
Thus, the chromosome length will be increased which in turn would result in the increase of problem complexity. For this reason, it is essential to find some techniques to decrease the chromosome length. Some of applicable techniques are: Converting the population of chromosomes to multi population. Fixing some of the parameters in any chromosome that are not very important or critical. Assuming the parameters of any chromosome as diagonal matrix.
In the first technique, the optimization of the whole parameters will not be done simultaneously and probably it will not result in the optimum result. The second technique is incoherence with the desired aim (adapting the controller parameters under the stability condition for different stroked patients and for different states of progression in the therapy process). Therefore, the third technique is applied in this study. And our mechanism for parent selection is truncation selection with this defined threshold in any generation. T = average of Fitness_Function Crossover operator is defined as one point crossover and point of crossover is selected randomly.
The flowchart of suggested algorithm is shown in Figure 3.

Results and discussion
For implementation of the suggested algorithm on the planar 2-DOF robot described in previous sections, there are several requirements (in terms of position, joint torques, impedance parameters) needed to control the manipulator (MP) as the sequel:

1-Desired position and velocity (trajectory) of MP and
ΔP d obtained from the physiotherapist. 2-Finding the appropriate joint variables with desired trajectory based on IK implemented by NN. 3-Optimization of impedance control parameters using GA in order to determine the required torques.
All these requirements were discussed in the previous sections.
For this purpose, a physiotherapy simple mode and its trajectory are defined which are shown in Figure 4.
The angles and velocities of joints for this robot are planned in three phases: 1) Horizontal trajectory from (x٫0) to (x r ٫0) with the speed of 1 m/s where x is the leg length in maximum extension and x r is the distance between hip source and manipulator in minimum flexion in x direction (it is marked as the 1st phase in Figure 4). 2) Circular trajectory around hip from (x r ٫0) to (0٫y r ) with the speed of 1 rad/s where y r is the distance between source and manipulator in minimum flexion in y direction (it is marked as 2nd phase in Figure 4). 3) Vertical trajectory from (0٫y r ) to (0٫y) with speed of 1 m/s where y is the leg length in maximum extension (in it is marked as 3rd phase in Figure 4). Assuming: The velocities in three phases will be: Angles of the joints in this physiotherapy mode are obtained based on the equations of the inverse kinematic (IK) problem in the suggested 2-DOF planar robot: Taking into account the following: And they are shown in Figure 5. This figure depicts three phases in desired trajectory described in Figure 4.
Where the range of joint2 is complementary of q 2 in Eq. (2). Now, the proposed MLP neural network is used for solving the IK problem. The weight and bias of the MLP neural network for joint1 approximation are obtained after training for 100 epochs which are shown in Table 1. In this table w{1,1} and bi{1} are the weights and biases of layer (1), respectively and w{2,1} and bi{2} are the weights and bias of layer (2), respectively.
The estimated values are shown in Figures 6 and 7, respectively.
The impedance parameters K d , D d ,M d are initially selected by a trial and error method subject to stability conditions and then they are tuned using GA algorithm. These parameters are chosen as below: Figure 4 Trajectory in three phases. The forces and torques that will be used for moving manipulator on the desired path (for 140 points of trajectory in 4 sec) are shown in Figures 8 and 9, respectively.
The evident peaks in Figure 9 denote the complementary movement.
The final optimized control parameters based on related torques and forces obtained as follows: Now, the obtained torques from Figure 9 are used for moving the robot. Figures 10 and 11 show the actual (q) and desired joint angles. It should be noted that the desired joint angles were approximated by MLP neural network that have been already described.
According to Figures 10 and 11 the deviation or difference between desired and actual variables is large for the first stages which are not suitable for rehabilitation without supervision. The deviation becomes smaller and converges to zero with the progress of simulation steps.
The difference between the desired and actual trajectory is shown in Figure 12.   Figure 13.
And the optimal values for GA parameters are obtained as: As the Figure 13 shows, the best control parameters obtained after 100 generations and the fitness function value is: -15.287.

Conclusions
In this study, a therapeutic exercise planar 2-DOF robot was designed and controlled for lower-limb  rehabilitation. The robot manipulator was controlled by combination of hybrid and adaptive controls. Some safety factors and stability constraints were defined and obtained. The robot is stopped when the safety factors are not satisfied. Kinematics of robot is estimated by an MLP neural network and proper control parameters are achieved using GA optimization.
The advantages of the proposed algorithm can be classified as the following: 1. The system is capable of learning the action of the physiotherapist for each patient and imitating this behavior in the absence of a physiotherapist that can be called robotherapy.    2. Generation of the source path is completely deliberative and it is done in accordance with the patient's condition and the therapy's duration. In this research, the source path was specified after various efforts such as visiting the specialists of the physiotherapy and observing several sessions in that section to completely gather the whole required information. 3. The neural network identifiers were used for solving the inverse kinematic of robot. The first idea for using NN is to cope with a non-linear identification problem and the second, more important one, is that the patient's joints controlling system can be probably replaced by the artificial neural network. 4. Safety is guaranteed since some of the controller parameters can be adapted under the stability condition for different stroked patients and for different states of progression in the therapy process. 5. To reduce the complexity of optimization of control parameters, genetic evolution method was used. A different aspect of the defined chromosomes in the suggested algorithm in comparison to conventional methods is that they are defined as matrices not as vectors which were placed because of the abundance of DOF for a system.
In comparison to other related works, some other remarkable issues can be added as follows: 6. The work places that are needed for LOKOMAT [5] and LOPES [7] must be in a large room while the whole place that is needed for manufactured robot is 1 m 2 in maximum. Moreover, the cost of rehabilitation with LOKOMAT is very high. 7. The number of DOF in ALEX [6] is very high but in 2-DOF planar robot it is limited to 2. 8. Only two parameters regarding the patient are used for starting the rehabilitation including mass of patient and ability in posture of ankle on the MP. The other parameters such as patient muscles, length and posture of whole body are not required.