Transparent composite model for large scale image/video processing

This paper aims to tackle theoretical modeling and dimension reduction, two fundamental issues in large scale image/video data processing, together, by proposing a transparent composite model (TCM) for transformed image/video data. Specifically, to handle the heavy tail phenomenon commonly seen in Discrete Cosine Transform (DCT) coefficients of image/video data, a TCM first separates the tail of a sequence of DCT coefficients from the main body of the sequence. Then, a parametric distribution is used to model the main body while a uniform distribution is used to model the tail. Efficient online algorithms for establishing a TCM are proposed and proved to converge exponentially fast, which suits large-scale image/video data processing. It is also demonstrated that a TCM has an inherent non-linear data reduction capability - DCT coefficients of an image in the heavy tail identified by a TCM reveal some unique global features of the image while being insignificant statistically. This, together with its fast convergence, makes the proposed model a desirable choice for modeling DCT coefficients in large-scale image/video applications, such as online quantization design, entropy coding design, and image/video analytics in Big Data.