End-to-end rate distortion optimization for H.264/AVC in error prone environment

To efficiently transmit H.264/AVC coded video over error prone channel, an end-to-end rate distortion optimized motion estimation is proposed. The end-to-end pixel level distortion of transmitted frame is estimated at the encoder, and then the distortion of reference frame area is weighted and combined with the conventional Lagrangian cost of motion estimation to form an enhanced cost function. By minimizing this cost function, the optimal motion vector is searched which lead to certain inter code mode of higher RD efficiency compared to intra code mode, while avoid the well known error propagation effects, probably with degraded prediction accuracy. The weight of the reference frame distortion is adapted on a macroblock basis under a constraint to the loss in prediction accuracy. The presented technique is verified within the H.264/AVC standard based on common test conditions.