A new cascaded projection pursuit network for nonlinear regression

Cascaded correlation is a popular supervised learning architecture that dynamically grows layers of hidden neurons of fixed nonlinear activations (e.g., sigmoids), so that the network topology (size, depth) can be efficiently determined. Similar to a cascaded correlation learning network (CCLN), a projection pursuit learning network (PPLN) also dynamically grows the hidden neurons. Unlike a CCLN where cascaded connections from the existing hidden units to the new candidate hidden unit are required to establish high-order nonlinearity in approximating the residual error, a PPLN approximates the high-order nonlinearity by using trainable nonlinear nodal activation functions (e.g., hermite polynomials). To relax the necessity of predefined smoothness of nonlinearity (e.g., the order of hermite polynomials) in a PPLN, the authors propose a new learning network, called a cascaded projection pursuit network (CPPN), which combines both advantages of a PPLN and a CCLN. The training strategy of a CPPN is similar to that of PPLN and the added cascaded connections allow the CPNN to better capture the high order features without the necessity of proper selection of orders of polynomials. Simulation results show that a CPPN is suitable for nonlinear regression and outperforms a PPLN when the polynomial order used in the activation functions is less than the order of target functions