Structural similarity optimal MB layer rate control for H.264

Conventional Rate Control (RC) schemes for video coding mostly take objective metric as distortion measurement, which can not achieve optimal subjective quality. This work applies Structural Similarity (SSIM) based subjective distortion to rate distortion optimization and RC in H.264, and proposes a SSIM optimal macroblock (MB) layer RC scheme. First, a SSIM quadratic distortion model is proposed based on extensive experiments and theoretical analysis. Then an improved quadratic Rate Quantization (R-Q) model is combined to obtain the solution of SSIM optimal MB layer quantization step (Qstep) by Lagrange multiplier method. Experimental results show that the proposed scheme preserves more image structural information and thus acquires better subjective quality compared with objective metric optimal MB layer RC JVT-O016 and classic JVT-G012.