A structure optimization algorithm of neural networks for large-scale data sets

Over the past several decades, neural networks have evolved into powerful computation systems, which are able to learn complex nonlinear input-output relationship from data. However, the structure optimization problem of neural network is a big challenge for processing huge-volumed, diversified and uncertain data. This paper focuses on this problem and introduces a network pruning algorithm based on sparse representation, termed SRP. The proposed approach starts with a large network, then selects important hidden neurons from the original structure using a forward selection criterion that minimizes the residual output error. Furthermore, the presented algorithm has no constraints on the network type. The efficiency of the proposed approach is evaluated based on several benchmark data sets. We also evaluate the performance of the proposed algorithm on a real-world application of individual travel mode choice. The experimental results have shown that SRP performs favorably compared to alternative approaches.