Optimal pruning of feedforward neural networks based upon the Schmidt procedure

A common way of designing feedforward networks is to obtain a large network and then to prune less useful hidden units. Here, two non-heuristic pruning algorithms are derived from the Schmidt procedure. In both, orthonormal systems of basis functions are found, ordered, pruned, and mapped back to the original network. In the first algorithm, the orthonormal basis functions are found and ordered one at a time. In optimal pruning, the best subset of orthonormal basis functions is found for each size network. Simulation results are shown.