Pruning with replacement and automatic distance metric detection in limited general regression neural networks

In this paper, we propose a limited general regression neural network (LGRNN) for embedded systems. The LGRNN is an improved version of general regression neural network that continues incremental learning under a fixed number of hidden units. Initially, the LGRNN learns new samples incrementally by allocating new hidden units. If the number of hidden units reaches the upper bound, the LGRNN has to remove one useless hidden unit to learn a new sample. However, there are cases in which the adverse effects of removing a useless unit are greater than the positive effects of learning the new sample. In this case, the LGRNN should refrain from learning the new sample. To achieve this, the LGRNN predicts the effects of several learning options (e.g., ignore or learning) before the learning process begins, and chooses the best learning option to be executed. Meanwhile, the LGRNN optimizes a hyper parameter for determining the distance metric automatically. Experimental results show that the method successfully reduces errors even when the number of hidden units is limited to a certain upper bound.