Fast Inter-Mode Decision in an H.264/AVC Encoder Using Mode and Lagrangian Cost Correlation

In this paper we present a novel algorithm to speed up the inter-mode decision process for the H.264/AVC encoding. The proposed inter-mode decision scheme determines the best coding mode of a given macroblock (MB) by predicting the best mode from neighboring MBs in time and in space and by estimating its rate-distortion (RD) cost from the MB in the previous frame. The performance of the proposed algorithm is evaluated in metrics such as the encoding time, the average peak signal-to-noise ratio and the coding bit-rate for test sequences. Simulation results demonstrate that the proposed algorithm can determine the best mode using only one or two rate-distortion cost computations for about half of the MBs resulting in up to 56% total encoding time reduction with on average 2.4% of bit rate increase at the same PSNR compared to H.264/AVC JM 12.1.