Learning in a layered neural network by the Hamiltonian algorithm

This paper deals with a layered neural network with a single hidden layer that consists of units of a hypersphere discrimination type. By permitting the hypersphere parameters, that is, the radius and the center vector, to take sufficiently large values, a unit of this type is enabled to efficiently extract not only a local feature but also a global feature in the feature space. To determine the size of the hidden layer, a network growing technique is used. The cost function of the network is in general apt to have many local minima and spacious flat regions. As a teaming method suitable for such a cost function, we introduce the Hamiltonian algorithm, which is a global minimum search technique proposed by Shinjo. Computer experiments on function approximation problems indicate that the proposed method has sufficiently good performance from the viewpoints of mapping ability, model complexity, and computational cost for teaming.