Visualizing Support Vectors and topological data mapping for improved generalization capabilities

This paper presents a method to improve generalization capabilities of supervised neural networks based on topological data mapping used in Counter Propagation Networks (CPNs). Using topological data mapping on CPNs the method presented herein provides advantages to interpolate new data in sparse areas that exist among categories and to remove overlapping or conflicting data in original training data. Moreover, our method can control the number of training data by changing the size of the category map according to a problem to be solved. As a type of supervised neural networks combined with our method, we select Support Vector Machines (SVMs), which are attractive as learning algorithms having high generalization capabilities to be mapped to a high-dimensional space using kernel functions. We applied our method to classification problems of two-dimensional datasets for evaluation of basic characteristics of our method. Topological data mapping based compression of original training data induces resolution of conflict among data and reducing the number of Support Vectors (SVs) that are absorbed as soft margins. The classification results show that decision boundaries are changed and that generalization capabilities are improved using our method. Moreover, we applied our method to face recognition under various illumination conditions using the Yale Face Database B. The results indicate that our method provides not only improved generalization capabilities, but also visualizes spatial distributions of SVs on a category map.