Shape of learning surface in multilayer neural networks

Summary form only given. The shape of a learning surface in a multilayer neural network model was analyzed. The learning surface is composed of plural elemental surfaces. The analyses show that there are two main features in the shape of an elemental surface. One is a curved `valley´ which is like a narrow groove. The other is a `plateau´ with a gentle slope. The shape of the learning surface also has two similar features peculiar to the elemental surfaces. The bottom of the valley of an elemental surface is a set of weights which are solutions of a nonlinear equation corresponding to an input-output training pattern. This solution set of weights makes a supersurface called a memory surface in high-dimensional weight space. The learning surface shows how plural memory surfaces are piled up on each other