A novel algorithm to fast mode decision with consideration about video texture in H.264

H.264 employs 7 different size block types for motion estimation that can significantly improve the coding performance compared with the previous video coding standards. However, H.264 requires extremely high computation with the R-D optimized decision since so many prediction modes are used. In this paper, a novel inter mode decision algorithm (NIMDA) is proposed that utilizes SADs of each 4X4 block and texture characteristic to reduce the candidate mode set after the 16 X16 prediction mode is tested. The simulation results show that the proposed algorithm reduces the entire encoding time by 64.52% with only negligible coding loss.