Linear discriminants, logic functions, backpropagation, and improved convergence

A modified learning algorithm for BP (backpropagation) neural networks is presented based on the interpretation of a neuron in an (N+1)-dimensional space, R^N+1, and the analysis of how a multilayered network performs a classification task by collective use of the B (boundary) neurons in the first layer. The role of B neurons and L (logic) neurons is discussed. A B neuron represents a linear boundary in the input space R^N. An L neuron in the second layer defines in R^N a convex piecewise linear boundary that is formed by a set of line segments corresponding to connected B neurons. An L neuron in the third layer defines in R^N a complicated boundary that can have both convex parts and concave parts. Each subboundary of it corresponds to an L neuron in the second layer. The nonlinear function does not change the structure of a boundary but smooths the angular vertices of a boundary. It is shown that for the two-class problems with convex boundaries, the nonlinear function in an L neuron can be replaced with a step and the weights of the L neuron can be fixed. Computer simulation shows that this algorithm can learn more quickly than the ordinary BP algorithm, even in cases where the ordinary BP algorithm will fail