Heterogeneous Multi-column ConvNets with a Fusion Framework for Object Recognition

The purpose of this paper is to investigate heterogeneous multi-column ConvNets (MCCNN) and fusion methods for them. We first construct heterogeneous MCCNN by combining ConvNets with different structures. We then use different fusion methods to check their performances to find out the effect of fusion methods for MCCNN. We also propose a novel sliding window based fusion framework which defines a specific subset of columns to be picked up from MCCNN for fusion. Two different strategies (exhaustive sliding window and sliding window from training) are investigated to determine the best performance of the fusion process. We tested the heterogeneous MCCNN and sliding window fusion on the MNIST dataset for optical character recognition. Experiments show that MCCNN improved the accuracy of recognition compared with a single column of ConvNets. Moreover, sliding window fusion is a more generalized fusion method and consistently achieves better results compared with the traditional fusion methods. We also tested the MCCNN and sliding window fusion on CIFAR-10 and Caltech-256 datasets. We achieved superior results compared to existing state-of-the-art techniques.