Fast intra prediction mode decision for H.264/AVC based on edge direction detection

The rate-distortion optimization (RDO) technique is employed in H.264/AVC intra frame coding to achieve the best rate-distortion performance. But the computational complexity is very high because it encodes the current block by exhaustively searching all possible modes to select the best one. To reduce the computational complexity we propose a new fast algorithm based on edge direction detection (EDD) to reduce the number of candidate modes for further RDO calculation. The proposed algorithm can accurately detect the edge direction of different blocks to reduce the number of candidate modes for further RDO calculation. The simulation results show that the proposed algorithm contributes to the bit-rate reduction compared to that of previous algorithms and saves around 62% of the encoding time with negligible PSNR degradation.