Implicit softmax transforms for dimensionality reduction

This paper develops implicit softmax transforms (IST) which are dimensionality reducing transforms that are defined implicitly by minimisation of a weighted sum of Kullback-Leib- ler distances (WKL). The parameters of an IST are trained in combination with the parameters of the polynomial exponents of softmax functions. The resulting gradient of the WKL can be efficiently calculated and the computational effort scales well with the size of the training set. The paper compares IST´s to PCA and LDA in classification experiments with two different types of classifiers on three different datasets, two of them from the UCI machine learning repository. It is shown that IST´s outperform PCA and LDA in a majority of the cases. In one case reducing the dimension with an IST even gives an improvement over the high dimensional baseline system.