Rank Estimation and Redundancy Reduction of High-Dimensional Noisy Signals With Preservation of Rare Vectors

In this paper, we address the problem of redundancy-reduction of high-dimensional noisy signals that may contain anomaly (rare) vectors, which we wish to preserve. For example, when applying redundancy reduction techniques to hyperspectral images, it is essential to preserve anomaly pixels for target detection purposes. Since rare-vectors contribute weakly to the -norm of the signal as compared to the noise, -based criteria are unsatisfactory for obtaining a good representation of these vectors. The proposed approach combines and norms for both signal-subspace and rank determination and considers two aspects: One aspect deals with signal-subspace estimation aiming to minimize the maximum of data-residual -norms, denoted as , for a given rank conjecture. The other determines whether the rank conjecture is valid for the obtained signal-subspace by applying Extreme Value Theory results to model the distribution of the noise -norm. These two operations are performed alternately using a suboptimal greedy algorithm, which makes the proposed approach practically plausible. The algorithm was applied on both synthetically simulated data and on a real hyperspectral image producing better results than common -based methods.