A mode filtering algorithm for accelerating HEVC FME

In the high efficiency video coding (HEVC), motion estimation (ME) is an essential process for inter prediction. Compared to integer ME (IME), fractional ME (FME) occupies much more encoding time in the fast search case. To reduce the computational complexity of FME, we propose a fast algorithm based on mode filtering. In the proposal, according to results of IME and neighboring largest coding units (LCU), 1 or 2 levels are selected from the original 4-level quad-tree structure, and only the selected levels will process FME. Specifically, quad-tree separation is adopted to remove the interdependence between IME and FME. Experiments on a wide range of video sequences show that the proposed algorithm can reduce up to 42.9% and an average of 33.4% total encoding time for HEVC with little loss in PSNR and rate.