A partially recurrent mixture-of-experts model for task decomposition into temporal and static subtasks

Modular neural network architectures often require subtasks to be allocated to each of the component networks by the modeller. The mixture-of-experts model overcomes this limitation by employing a gating network to `discover´ an appropriate decomposition of the computation learnt during the training procedure. Hence, the mixture-of-experts model is a neural network which is capable of task decomposition. However, it is a static network in that it is incapable of temporal processing. We propose a dynamic version of the mixture-of-experts model by introducing recurrent links into the architecture, and investigate automatic task decomposition into temporal and static subtasks. The model successfully simulates development like the acquisition of quantification