Residual analysis based fast inter mode decision for H.264/AVC

H.264/AVC introduces the variable block size macroblock mode for motion estimation, which brings huge computational cost. In this paper, a novel fast inter macroblock mode decision algorithm for H.264/AVC has been proposed. The proposed algorithm evaluates the modes based on residual information. The residual is obtained after the motion search of P16 × 16 mode or P8 × 8 mode. Then the characteristic of the blocks are extracted by the defined conversion, and the complexity and similarity are calculated for the decision the next conducting mode. The mode with the minimal R-D cost is chosen as the final mode. The simulation results show that, the proposed algorithm achieves 65.95% time-saving, which is 1.5 times than that of other algorithm with less PSNR loss and bit rate increase.