SATD-Based Intra Mode Decision for H.264/AVC Video Coding

In this letter, a rate estimation technique using transform coefficient variance is proposed in the rate-distortion cost function for intra mode decision. The new cost function achieves better coding performance than the cost function using the ??-domain model. The new cost function brings out an average speed-up factor of 13 when compared with the rate-distortion optimization (RDO) mode decision, but with considerable degradation in rate-distortion performance. To improve the coding performance, we propose a sum of the absolute transformed differences (SATD)-based mode decision algorithm, which incorporates both SATD and the variance information into the RDO mode decision method. The results show that the SATD-based mode decision scheme can achieve a significant improvement in computation (with 21% of total encoding time, or a speed-up factor of 5) with negligible peak signal-to-noise ratio loss. The proposed algorithm is also compared with two other distinct algorithms , and the results indicate that the proposed algorithm outperforms these two edge detection algorithms in coding performance, as well as time saving.