SATD-based intramode decision for H.264/AVC video coding

In this paper, a rate estimation technique using the transform coefficient variance is proposed in the rate-distortion cost function for intra mode decision. The experiment shows that the rate estimation using the absolute transform coefficient variance can predict rates more accurate than the one using rho-domain model. The new cost function brings out an average speed-up factor of 13.8 when compared with the RDO mode decision, but with considerable degradation in the rate-distortion performance. To improve the coding performance, we propose a SATD-based mode decision algorithm, which incorporates both SATD and the variance information into the RDO mode decision method. The results show that the SATD-based mode decision scheme can achieve a significant improvement in computation with negligible PSNR loss.