Automatic construct of options in RL

The taboo state is introduced in environment to discovery sub-goal. Agent samples trajectories from starting state to goal state, which contain different bottlenecks. Then the different tasks are submitted to agent. According to whether the task is accomplished or not, the bottlenecks among them are discovered. The appropriate bottlenecks are selected as sub-goal of options to be constructed according to the adjacent relationship among them. Simultaneously agent can obtain the initial set and the policies of options. Grid-world tasks illustrate that the agent can automatically construct useful options online with the proposed method, which have capability of accelerating learning and the transference of knowledge among those similar learning tasks.