Nonnegative matrix factorization with maximum self-information on basis components

In this paper, we propose a novel method, called nonnegative matrix factorization with maximum self-information on basis components (NMFMSI), for learning informative, spatially localized and parts-based subspace representations of visual patterns. In addition to the nonnegativity constraint in the standard nonnegative matrix factorization model, a new objective function is defined to impose maximum self-information and smoothness constraints on the basis components and the encoding vectors, respectively. NMFMSI yields a set of basis features which not only allows an additive representation of data but also contains more information about the data. The self-information of a basis feature is closely related to its probability density. By using the Oja rules, an algorithm is presented for learning the nonnegative solutions of NMFMSI. Experimental results on the swimmer dataset and ORL face database demonstrate the advantages of NMFMSI.