Optimal transform formed by a combination of nonlinear operators: the case of data dimensionality reduction

In this paper, a new approach to constructing optimal nonlinear transforms of random vectors is proposed and justified. The proposed transform T_p is presented in the form of a sum with p terms where each term is interpreted as a particular rank-reduced transform. Moreover, terms in T_p are represented as a combination of three operations F_k, Q_k, and ψ_k with k=1,...,p. The prime idea is to determine F_k separately, for each k=1,...,p, from an associated rank-constrained minimization problem similar to that used in the Karhunen-Loe`ve transform (KLT). The operations Q_k and ψ_k are auxiliary for finding F_k. The contribution of each term in T_p improves the entire transform performance. A rigorous analysis of errors associated with the proposed transforms is given. It is shown that the proposed transform improves such characteristics of rank-reduced transforms as compression ratio and the accuracy of decompression and reduces required computational work. The Fourier series in Hilbert space, the Wiener filter, the KLT and the transforms presented in earlier author papers in the field are particular cases of the proposed method. Theoretical results are illustrated with numerical simulations.