A proposal of neural network architecture for non-linear function approximation

In this paper, a neural network architecture for non-linear function approximation is proposed. We point out problems in non-linear function approximation with traditional neural networks, that is, difficulty in analyzing internal representation, no reproducibility in function approximation due to the random scheme for weight initialization, and the insufficient generalization ability in learning without enough samples. Based on these considerations, we suggest three main improvements. The first is the design of a sigmoidal function with localized derivative. The second is a deterministic scheme for weight initialization. The third is an updating rule for weight parameters. Simulation results show beneficial characteristics of our proposed method; low approximation error at the beginning of iterative calculation, smooth convergence of error and its improvement for difficulty in analyzing internal representation.