Online learning of low dimensional strategies for high-level push recovery in bipedal humanoid robots

Bipedal humanoid robots will fall under unforeseen perturbations without active stabilization. Humans use dynamic full body behaviors in response to perturbations, and recent bipedal robot controllers for balancing are based upon human biomechanical responses. However these controllers rely on simplified physical models and accurate state information, making them less effective on physical robots in uncertain environments. In our previous work, we have proposed a hierarchical control architecture that learns from repeated trials to switch between low-level biomechanically-motivated strategies in response to perturbations. However in practice, it is hard to learn a complex strategy from limited number of trials available with physical robots. In this work, we focus on the very problem of efficiently learning the high-level push recovery strategy, using simulated models of the robot with different levels of abstraction, and finally the physical robot. From the state trajectory information generated using different models and a physical robot, we find a common low dimensional strategy for high level push recovery, which can be effectively learned in an online fashion from a small number of experimental trials on a physical robot. This learning approach is evaluated in physics-based simulations as well as on a small humanoid robot. Our results demonstrate how well this method stabilizes the robot during walking and whole body manipulation tasks.