Relaxing in a warped space: an effect due to the cooperation of static and dynamical neurons

This paper proposes a module-based neural network composed of static and dynamical neurons, and discusses what effect can be produces by the integration of mapping and relaxation. The proposed network can be obtained when a specially-designed total energy function is minimized. Although the derivation process is similar to the case of the well-known Hopfield network, state variables in the network included, in addition to the addition to the output and the potential of dynamical neurons, another type of state variable converted from the direct output of dynamical neurons using a mapping function. If we suppose a layered network with only static neurons corresponding to the mapping function, the layered network comprises a forward subnet and a backward subnet; connection weights in the forward and backward subnets are modified based on propagated error signals through the backward subnet, and, at the same time, the final output of the backward subnet is utilized for overall network-dynamics. As a result, the proposed network offers a framework in which relaxation can be carried out in a warped space due to the cooperation of static and dynamical neurons. Furthermore, it gives an interpretation for the backpropagated error signals in the case of delta-rule based learning; although the backward subnet for the calculating the delta values is usually assumed to be virtual, it must actually exist for network relaxation in the proposed network.