Hybrid learning strategy to solve pendulum swing-up problem for real hardware

In this paper, we propose a machine learning strategy to obtain the optimal controller for actual machine using hybrid platforms; real hardware and simulator. A simulator consists of the neural networks which directly can learn actual behaviors of the latest hardware and emulates them without physical modeling. On the other hand, the controller of the hardware is trained with the simulator by the reinforcement learning method to realize the optimal control for the target task, and applied to the real hardware. Then, as long as the iteration of these processes is simultaneously performed, the system can automatically generate the optimal controller without any works even when hardware constitution is changed or switched. In this manner, the real hardware and the simulator affect each other to make the system adaptable. Furthermore, in the processes of sampling and supplying hardware data, we put a buffering component. It keeps the latest data of the hardware and supplies non-biased data to the simulator. As an example of the proposal method, we pick up the pendulum swing-up problem. In the experiments, firstly, the optimization process performs step by step for the initial hardware constitution and the basic idea of the method is evaluated. Afterward, by changing a pendulum, we confirm system can autonomously generate the new optimal controller for the real hardware without any human operations.