Network grafting: Transferring learned features from trained neural networks

The degree of abundance of labeled training data is an important factor in determining the performance of supervised machine learning systems. However, in some applications, labeled data are either costly to collect or easily outdated, resulting in poor generalization of trained machine learners. Nonetheless, there are often related domains where large corpuses of labeled data can be easily obtained. Therefore, we propose a new transfer learning algorithm to adapt a neural network trained on such a related domain to the target domain by grafting additional nodes onto its hidden layer. This neural grafting method is capable of transfering the knowledge embedded in the structures of the trained neural network to the problem in the target domain. Experiments on synthesized and real data sets show that grafted networks achieve good performance with very small amounts of data from the target domain. Compared with existing transfer learning techniques, the proposed neural grafting is easy to tune and computationally simple, with superior performance.