Feedback Error Learning for Controlling 1-DOF Joint Angle: Model Simulation and Experiment

Kenji Kurosawa*, Ryoko Futami, Takashi Watanabe, Nozomu Hoshimiya

Tohoku University, Sendai, JAPAN

Introduction

To date, many FES control strategies have been proposed in the literature. They are basically categorized into three types: open-loop, closed-loop, and hybrid controllers. From the viewpoint of system simplicity and self-maintainability by patients, open-loop control is acceptable. However, open-loop controllers need initial calibration and periodic adjustment of stimulation levels and timing for individual patients, which must be done by clinicians. These adjustments can take significant time to complete. Thus, we propose adopting the "Feedback Error Learning (FEL)" scheme [1,2] in an FES controller. This controller is expected to simplify and shorten the clinical adjustment tasks, although appropriate sensor systems are needed in the learning phase. The FEL controller is regarded as a hybrid controller including a feedforward and a feedback controller.

An able bodied subject’s wrist angle (the palmar/dorsi- flexion) was controlled with the FEL controller to track a desired trajectory by stimulating two muscles (ECRL/ECRB and FCU). Before conducting the experiments, some parameters in the controller were determined in advance by computer simulation using a forward model of the wrist. The model had been prepared by training an artificial neural network so as to imitate the response of the subject’s wrist angle to electrical stimulation. The results showed that the FEL controller performed better than the conventional PID controller in the evaluation of an average error for a fast movement (cycle period: 2s). Furthermore, the performance of the controller was improved when an Inverse Dynamics Model (IDM) in the FEL controller had been trained in advance with the forward model. Only a few trials of stimulation were required to track the trajectory with a small error value.

Methods

Fig.1 shows the block diagram of the FEL (Feedback Error Learning) controller, which combines the feedforward controller (Inverse Dynamics Model; IDM) and the feedback controller. The feedforward controller enables fast movement without delay, whereas the feedback controller has the advantage of high stability and high performance for disturbance compensation. The FEL controller has both advantages. The inverse dynamics model is trained on-line using the output of the feedback controller. Therefore, both the control and the training of the IDM are carried out at the same time (i.e. the IDM captures the inverse characteristics of limbs while controlling them). The feedback controller acts mainly at the beginning of the control, then gradually shifts the role to the feedforward controller as the learning progresses. In this study, the four-layered perceptron was used to express the IDM, which was learnt by the back-propagation technique. The output of the feedback controller was used as the error signal for back-propagation. The error signal was back propagated through the IDM at a hundred milli-seconds past state to cancel out the delay present in electrically stimulated muscles. The initial values of connection weights in the IDM were determined by a random number generator. The traditional PID controller was adopted as the feedback controller; however, the PID controller was improved to be able to control redundant musculoskeletal systems [3].

Experiments were conducted with the controller to evaluate its control and learning performance. Before the experiments, some parameters in the controller, such as the number of neurons in the hidden layer of the IDM and the learning coefficient for back-propagation, were determined in advance by computer simulation. A forward model of the subject’s wrist angle response to electrical stimulation was used in the simulation study. The forward model had been prepared by training a neural network using the measured response of a subject’s wrist. The palmar/dorsi-flexion angle was controlled by stimulating ECR (ECRL & ECRB) and FCU. The subject was an able bodied male who had enough experience with electrical stimulation. His ECR and FCU were stimulated simultaneously through a pair of Ag/AgCl surface electrodes. The laboratory FES system incorporating the 3-D position-orientation measurement system (FASTRAK, Polhemus Inc.) was used to generate stimulus pulses and to measure the wrist angle [4]. The pulse amplitude was modulated in the range of 0-25mA, whereas the pulse width and the frequency were fixed at 200 microseconds and 20Hz,

Figure 1. The block diagram of the FEL (Feedback Error Learning) controller.

respectively. The subject was seated on a chair with his left arm hanging (i.e. in the gravitational direction); the shoulder, the elbow and the hand were free. The subject was instructed to relax his left arm and hand as much as possible. The neutral angle of the wrist was defined at this posture. The desired trajectory was ten reciprocating motion of 40 degrees in amplitude and 2s in cycle period. The control by the FEL was repeated up to 40 times. The control performance was evaluated (mean error) by comparison with the PID controller alone. Control by the FEL in which the IDM had been well trained with the forward model was also carried out.

Results

One of the results is shown in Fig.2. Fig.2 (a) is the result of the first trial (the iteration number is 1) and Fig.2 (b) is the 24th trial. The upper graphs show the time courses of the wrist angle, and the lower graphs describe the stimulation current. When the PID controller was used alone, the results were almost the same as Fig.2 (a), in which the average error was 7.3[degree]. It can be seen that there is a large delay in the measured trajectory on the first iteration (Fig.2 (a)). After 23 training cycles, the delay decreased as observed in Fig.2 (b), resulting in a 55% decrease of the average error. It was thought that the performance improved because the IDM successfully obtained the inverse characteristics of the subject’s wrist. The change in the error and the power is shown in Fig.2 (c). In the graph, power is the ratio of the output power of the IDM to the total power of both the IDM and the PID controller. The error decreased and the power ratio increased with increased iterations. The change of the power shows that the main controller was changed gradually from the PID to the IDM. At the 24th iteration, the IDM held 80% of the control. These results show that the FEL controller functioned as expected and its control performance was superior to the conventional PID controller. However, the error increased when the iteration number exceeded 30.

Figure 2. Results of the wrist angle control: (a) The time course of the wrist angle (upper) and the stimulation (lower) at the first trial (The iteration number is 1). (b) The time course of the wrist angle and the stimulation at the 24th trial. (c) The change of the average tracking error and the power ratio (Power of the IDM / Total Power). “random” is the result when the initial connection weight values in the IDM were determined by a random number generator. “trained”: the IDM was trained in advance with the forward model.

The gray lines labeled as “trained” in Fig.2 (c) show the result with the model-trained IDM. Note that the forward model was not precise as several days had passed after the model was developed and the surface electrodes had since been replaced. Nevertheless, the error was small and the power ratio was high from the first iteration. Moreover, the error was minimized in all trials with high power ratio after a few additional iterations.

Discussion

The experimental results show that the delay in movement and the mean tracking error decreased to less than half of the value of that obtained with traditional PID controllers, after only 20 to 30 iterations. Only a few iterations were required to control with the smallest error when the IDM had been trained in advance with the artificial forward model. The required iteration number is so small in both the cases that the controller is practical for clinical applications. In the experiment, the forward model was not precise because several days had passed after the model had been developed. The results may be better if the control experiments are conducted soon after the development of the forward model. Moreover, adjustment of the learning coefficient will be required for smoother convergence of the error.

When multiple joints are controlled simultaneously (e.g. reaching movements, etc.), delay in one joint movement yields failure in tracking the trajectories of the fingertips because of the kinematics. Thus, the controller as described in this paper, which can control limbs with little delay, is required for accurate movement control of paralyzed limbs by FES. Furthermore, the controller can be applied for controlling not only upper limbs but also lower limbs, even though the FEL controller was evaluated on the wrist angle control in this paper. The evaluation of the controller on a 2-degree of freedom joint (the dorsi/palmar-flexion and the radial/ulnar-flexion of the wrist) is our next research interest.

References

[1] Kawato, M., Feedback-error-learning neural network for supervised motor learning. Adv. Neural Comput., 1990. p.365-372.

[2] Vojislav D. Kalanovic, Dejan Popovic, and Nils T. Skaug, Feedback Error Learning Neural Network for Trans-Femoral Prosthesis. IEEE Trans Rehab Eng, 2000. 8(1): p.71-80.

[3] Watanabe, T., Iibuchi, K., Kurosawa, K., Futami, R., and Hoshimiya, N., A Method for Solving Ill-Posed Problem in Multichannel Closed-Loop FES Control. Proc. 6th Annual IFESS Conf., 2001

[4] Kurosawa, K., Watanabe, T., Futami, R., Hoshimiya, N., and Handa, Y., Development of a Closed-loop FES System Using a 3-D Magnetic Position and Orientation Measurement System. J. Automatic Control, 2002. 12: p.23-30.

Acknowledgments: This study was partly supported by The Japan Society for the Promotion of Science under the Bilateral Program, Takayanagi Foundation for Electronics Science and Technology, and Miyagi Joint-Research Project for Regional Intensive. We would like to thank these organizations.