Adaptive rate distortion optimisation for H.264 intra coding

Rate distortion optimisation (RDO), which is the best performing mode decision method for H.264, does not consider dependence between neighbouring blocks, because calculating rate distortion (RD) costs for all possible mode sets requires too many computations. However, in H.264 intra coding, RD performance of the current block is affected by reference pixels in neighbouring blocks. Thus, considering dependence between neighbouring blocks can enhance RD performance in mode decision. Proposed is an adaptive RDO for H.264 intra coding. To consider block dependence with only a few more computations, a mode decision criterion is proposed for 14times4 using the partial boundary sum of the squared differences. Experimental results show that the proposed method improves RD performance compared to RDO.