Paralysis free fast learning: smart neural nets

We introduce fast learning fully connected feedforward smart neural nets by avoiding the use of sigmoid nonlinear function driven conventional neurons. We achieve this by introducing what we call the “smart neurons”. The smart neurons together with the linear ADALINEs are used to construct the fast learning smart neural nets. The smart neurons alone are used to form the hidden layers of the smart neural net. The output layer of the smart neural net is constructed by using the linear ADALINEs alone. Like the conventional neural nets, the smart neural nets can be trained using the regular innovation backpropagation algorithm. We compare the performance of the smart neural nets against the conventional neural nets. It is shown that the smart neural nets learn extremely faster than the conventional neural nets. Unlike the conventional neural nets, the smart neural nets proposed here can learn without ever becoming paralysed. The smart neural nets also behave well during the learning. In addition, we show that much more efficient and fast learning neural nets can be built by avoiding the conventional neurons altogether