Discovering sub-goals from layered state space

Task decomposition is an effective approach to accelerate reinforcement learning, but how agent can discover autonomously sub-goals remains an open problem. We propose an approach to make agent can take advantage of state trajectories it learned form training episodes to arrange states in different layers according to the shortest distance of them from the goal state, then to reach these state layers with different distance from the goal state could be the sub-goals for agent achieving the goal state eventually. Compared with others, our approach has two advantages: 1) the discovery of sub-goals is performed by agent itself without human´s aid; 2) the approach is robust, which is not limited to specific problem state space, but is applicable widely. The experiments on Grid-World show the applicability, effectiveness and robustness of our approach.