Memory capacity bound and threshold optimization in recurrent neural network with variable hysteresis threshold

The authors propose an asymmetrical associative neural network (NN) using variable hysteresis threshold and its learning and association algorithms. It can drastically improve the noise performance, i.e. insensitivity to noise. The memory capacity bound and threshold optimization in this associative NN are further discussed. Binary random patterns are considered. First, relation between the number of patterns and the number of iterations is investigated. The latter gradually increases until some number of patterns. After that, it suddenly increases. This is a very peculiar phenomenon. This turning point gives the memory capacity bound, that is about 1.56N, where N is the number of units. Next, the threshold optimization is discussed. Relation between threshold and noise performance, and effects of connection weight distribution on noise performance are theoretically discussed. Based on these results, a ratio of step-size and the threshold is optimized to be 0.5/(N_p-1), where N_P is the number of units on the pattern. Statistical simulation demonstrates the efficiency of the proposed methods.