Analysis of quadratic R-D model in H.264/AVC video coding

Due to the high performance in the rate control of MPEG-4, the quadratic Rate-Distortion model has been widely proved. The rate control module in the reference software of H.264/AVC directly inherits the quadratic model. However, the introduction of Rate Distortion Optimization (RDO) in H.264/AVC makes rate control more complex than previous standards. Based on theoretical derivation and extensive experiments, this paper analyzes the model error of the quadratic model and the prediction error of MAD in H.264/AVC. Simulation results demonstrate that our proposed linear MAD model achieves higher prediction accuracy than the quadratic mode does.