Automatic determination of optimal network topologies based on information theory and evolution

Presents a new approach to determine the optimal topology of multilayer perceptrons for a given learning task, based on information theory and evolution. Our method exploits the mutual information of the input-output relation to sort the units into a list with respect to their information content. Embedded in a evolutionary algorithm, a mutation operator is proposed which removes or adds input units from given networks based on their ranking. The power of the approach is demonstrated on several benchmarks. We conclude that using an evolutionary algorithm as a framework in conjunction with intelligent mutation operators is concurrently the most efficient optimization technique with regard to network size and performance as well as scalability.