Evaluating the performance of nonnegative matrix factorization for constructing semantic spaces: Comparison to latent semantic analysis

This study examines the ability of nonnegative matrix factorization (NMF) as a method for constructing semantic spaces, in which the meaning of each word is represented by a high-dimensional vector. The performance of two tests (i.e., a multiple-choice synonym test and a word association test) is compared between NMF and latent semantic analysis (LSA), which is the most popular method for constructing semantic spaces. As a result, it was found that NMF did not outperform LSA in either test. This finding indicates that NMF is less effective in acquiring word meanings than expected in the literature; in other words, the finding provides evidence for the ability of LSA to represent semantic meanings. Some properties of NMF were also revealed with reference to its ability to represent word meanings; the random initialization was superior to the SVD-based initialization, and the Euclidean distance is more appropriate for the objective function of NMF than the KL-divergence. In addition, it was shown that the inner product was a more appropriate method for measuring the syntagmatic similarity in a semantic space model, while the cosine was a better method for computing the paradigmatic similarity.