Rate-distortion-complexity optimization of fast motion estimation in H.264/MPEG-4 AVC

This paper presents an operational method for optimizing integer motion estimation in real-time H.264 encoding with respect to the trade-off between rate-distortion and complexity. This three-parameter problem is converted into a more tractable two-parameter problem by a simple approximative elimination of either the rate or the distortion term by converting small differences in the term to be eliminated. This paper also presents an H.264-enhanced implementation of the fast EPZS motion estimation algorithm, which adds three early-stop criteria controlled by predefined thresholds allowing it to stop after 16 16 and 8 8 block types and for each additional reference frame. Results for interlaced SDTV material are presented with indications of the applicability of the operational method when applied to the presented motion estimation procedure. For example, a speed-up by a factor of 4 compared to a basic H.264-adapted EPZS implementation is achieved at only 1% increase in rate.