Unified Solution to Nonnegative Data Factorization Problems

In this paper, we restudy the non-convex data factorization problems (regularized or not, unsupervised or supervised), where the optimization is confined in the nonnegative orthant, and provide a unified convergency provable solution based on multiplicative nonnegative update rules. This solution is general for optimization problems with block-wisely quadratic objective functions, and thus direct update rules can be derived by skipping over the tedious specific procedure deduction process and algorithmic convergence proof. By taking this unified solution as a general template, we i) re-explain several existing nonnegative data factorization algorithms, ii) develop a variant of nonnegative matrix factorization formulation for handling out-of-sample data, and Hi) propose a new nonnegative data factorization algorithm, called correlated co-decomposition (CCD), to simultaneously factorize two feature spaces by exploring the inter-correlated information. Experiments on both face recognition and multi-label image annotation tasks demonstrate the wide applicability of the unified solution as well as the effectiveness of two proposed new algorithms.