Probability-based coding mode prediction for H.264/AVC

Fast mode decision (FMD) algorithm aims to reduce the computational complexity of video encoders, especially for H.264/AVC, and meanwhile keep the coding efficiency. It is necessary for coding high resolution sequences, such as standard definition (SD) or high definition (HD) sequences. In this paper, an efficient algorithm is proposed to speed up inter-mode decision for H.264/AVC. Firstly, for a macroblock (MB) being coded, with modes of neighboring blocks, a probability model is utilized to build up an optimal encoding mode list; after that, rate-distortion (RD) cost-based early termination is employed to skip unnecessary inter-modes. Simulation results demonstrate that, the proposed algorithm could achieve significant encoding time saving for QCIF/CIF and SD/HD sequences, meanwhile with almost the same RD performance as the original encoder.