Frame-layer H.264 rate control improvement using Lagrange multiplier and quantizer

The paper presents a frame-layer rate control for H.264 that computes the Lagrange multiplier (λ_MODE) for mode decision by using a quantization parameter (QP) which may be different from that used for encoding. At the same time, we also compare the bits produced by each mode with the average target bits of each macroblock (MB) in a frame to modify λ_MODE further. The objective of these measures is to produce bits as close to the frame target bits as possible. In order to obtain an accurate QP for a frame, we employ a complexity-based bit allocation scheme and a QP adjustment method. Simulation results show that the H.264 encoder, using the proposed algorithm, controls the bit rate better, achieves a visual quality improvement of about 0.5 dB better than that of the existing H.264 rate control method (JVT-G012), with a smaller visual quality variation.