The learning parsimony of projection pursuit and back-propagation networks

The learning parsimony is studied for two types of feedforward network learning methods for model-free regression problems. One is back-propagation learning (BPL) and the other is projection pursuit learning (PPL). Algorithmically, both the BPL and the PPL parametrically form projections of the data in directions determined from interconnection weights. However, unlike BPL, which uses a fixed set of nonlinear nodal functions to perform an explicit parametric estimate of all the weights simultaneously at each iteration, PPL non-parametrically estimates the unknown nonlinear mapping sequentially at each iteration. In terms of learning efficiency, both methods have comparable training speed while the PPL is more parsimonious