Improve discontinuous output in SpikeProp — Effective type of weight decay

In this paper we improve the input-output relationship of SpikeProp network[1], one type of spiking neural networks. Though the standard SpikeProp networks perform well on complex non-linear classification, it has a drawback: discontinuity problem, which is a behavior that small variation in input causes the output to change greatly. Previous work shows that weight decay is effective for this problem. In this paper, we discuss the effect of three types of weight decay. By simple experiments, we conclude that the squared type of weight decay works well on this problem, and that to reduce the absolute large weights is more effective for the problem than to reduce the number of weights of the network.