Automatically Discovering Hierarchies in Multi-agent Reinforcement Learning

It is difficult to automatically discovering hierarchies in multi-agent reinforcement learning. We consider an immune clustering approach for automatically discovering hierarchies in option learning framework. The leading agent generates an undirected edge-weighted topological graph of the environment state transitions based on the environment information explored by all agents. An immune clustering algorithm is then used to partition the state space. A second immune response algorithm is used to update the clusters when a new state being encountered later. Local strategies for reaching the different parts of the space are learned distributedly and added to the model in a form of options.