Neural networks with branch gates

A new architecture of neural networks (NNs) is proposed named neural networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalization within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.