Transformed-Domain Intra Mode Decision in H.264/AVC Encoder

In this paper, a fast mode decision method for intra prediction is proposed to reduce the H.264/AVC encoder complexity. By using the edge directions of the coding blocks to estimate the prediction modes, we propose a novel method dedicated to H.264/AVC to reduce the computation overhead for edge detection. The edge direction of each block is detected from only part of coefficients obtained in the transformed domain. Based on the estimated edge direction, we can speed up the intra mode decision process of each coding block for both luma and chroma components