Online learning of foot placement for balanced bipedal walking

Due to the high complexity of the humanoid body, and its inherently unstable inverted pendulum-like dynamics, the development of a robust and versatile walking controller proves to be a difficult task. Using machine learning algorithms with hardware in the loop is a promising way of achieving balanced and dynamic gaits. In this work, we propose an online learning technique that learns how to step onto a reference footstep location while maintaining the balance of a bipedal walker in the presence of disturbances. The ability to step with the help of a parametrized motion generator simplifies the learning problem to the low-dimensional space of footstep coordinates. To quickly adapt the produced step sizes from learned experience, we update an online-capable function approximator with a pendulum-cart motivated gradient function that incorporates the trade-off between maintaining balance and stepping onto a desired location. While our method is able to robustly learn suitable footstep locations without prior knowledge, we gain advantage from initializing the learning with an analytic controller and show experimentally that the learning process can further improve the capabilities of the robot.