High-Order Tensor Decomposition for Large-Scale Data Analysis

Higher-order tensor decomposition is a basis for many important data mining tasks and the efficient large-scale tensor decomposition algorithms will have positive impact on clustering, trend detection, and anomaly detection. In the paper, we develop a scalable and distributed version of the Tucker tensor decomposition, MR-T, using the Hadoop MapReduce framework. We avoid large matrix-matrix multiplication and exploit the sparsity of large data sets to minimize intermediate data and flops by sequentially computing the intermediate matrices and generating the intermediate tensor vector-wise.