Simultaneous search for all modes in multilinear models

Parallel factor (PARAFAC) analysis is an extension of a low rank decomposition to higher way arrays, usually called tensors. Most of existing methods are based on an alternating least square (ALS) algorithm that proceeds iteratively, and minimizes a criterion (that is usually quadratic) of the fit with respect to individual factors one by one. Convergence of this approach is known to be slow, if some of the factor contain nearly co-linear vectors. This problem can be partly alleviated by an enhanced line search (ELS) by Rajih et al. (2008). In this paper we show that the method originally proposed by Paatero (1997), consisting in optimization with respect to all modes simultaneously, can be simplified, and can far outperform the ALS-ELS in ill-conditioned data in all modes.