Fast rate-distortion optimized transform for Intra coding

In our previous work, the rate-distortion optimized transform (RDOT) is introduced for Intra coding, which is featured by the usage of multiple offline-trained transform matrix candidates. The proposed RDOT achieves remarkable coding gain for KTA Intra coding, while maintaining almost the same computational complexity at the decoder. However, at the encoder, the computational complexity is increased drastically by the expensive ratedistortion (R-D) optimized selection of transform matrix. To resolve this problem, in this paper, we propose a fast RDOT scheme using macroblock- and block-level R-D cost thresholding. With the proposed method, unnecessary mode trials and R-D evaluations of transform matrices can be efficiently skipped from the mode decision process. Extensive experimental results show that, with negligible coding performance degradation, about 88.9% of the total encoding time is saved by the proposed method.