ρ-domain rate-distortion optimal rate control for DCT-based video coders

Most real-time MPEG-like encoders are designed to perform in a constant bit rate (CBR) mode, in which large deviations from a target rate are avoided at any time instant because of video buffer rules set at the decoder. A rate-control method at the encoder ensures that by intelligent selection of encoding parameters, the video buffer constraints are met, while at the same time a best possible video quality is achieved. Almost all reported rate-control methods model the rate and distortion in terms of the quantization step size. A recently suggested alternative is to use ρ, the fraction of zeros among the quantized DCT coefficients (He, Z. and Mitra, S.K., IEEE Trans. on Circuits and Systems for Video Technology, vol.12, no.11, 2002). We address the problems associated with the ρ-domain rate-control approaches in the literature. In addition, we propose a robust MPEG-2 rate-control algorithm that aims to minimize picture quality variations. Our extensive analysis with several video sequences and using the MPEG-2 codec indicates that the proposed algorithm can achieve a more uniform visual quality, and provides a PSNR gain that ranges from 0.5 dB at 2 Mbps to 0.9 dB at 6 Mbps over TM5 on the average.