On DCT coefficient distribution in video coding using quad-tree structured partition

The recent image/video coding standards adopt a full quad-tree structured block partitioning with a number of different transform sizes. However, the statistical behavior of the transformed coefficients becomes more difficult to estimate than the previous standards while the single pdf might be enough for modeling the distribution in the previous standards. In this paper, we propose a novel probability density to model the distribution of the transformed coefficients obtained from different transform sizes. The proposed density model is mathematically derived, and the parameters are estimated from previously coded samples. Rate-Distortion (R-D) model is also established from the proposed density model. It is shown in experimental results that the proposed pdf is efficiently used for approximating the rate-distortion (R-D) relation when the different transform sizes are adapted for a coding.