A new intra prediction mode decision algorithm in H.264

Although the coding mode of non-RDO in H.264 greatly increases the execution speed of intra prediction, shorter execution time is required. A new intra prediction mode decision algorithm is proposed. First, the intra prediction mode of two chroma macroblocks of same location is only decided by one chroma block. Second, the idea of low resolution and SAD criterion are exploited to select the best 16 times 16 intra prediction mode of luma macroblock quickly, and the number of candidates for the intra prediction modes of 4 times 4 subblock of luma macroblock is reduced by the best 16 times 16 intra prediction mode and block location. Then the coding type and the prediction mode of the luma macroblock are decided by flexible cost comparison. Simulation results show that the proposed algorithm greatly reduce computational load of the intra prediction mode decision with negligible loss of PSNR and little increase of bite-rate.