Joint Entropy Maximization in the Kernel-Based Linear Manifold Topographic Map

This paper introduces the kernel-based linear manifold topographic map and an information theoretic algorithm developed for its learning. The kernels represent lower dimensional local linear manifolds in a data space, and are defined in an optimal manner when special Gaussian input densities are assumed. The kernel parameters are adapted to maximize the joint entropy of the neuron outputs of the map. This is fulfilled by applying stochastic gradient ascent to the differential entropy of each neuron output and using competition between the neurons of the map. Topology preserving property is also possible by considering neighborhood functions. The proposed model can be considered as an improved version of the ASSOM network which maintains the ASSOM advantages while avoiding its limitations.