Categorization for fast intra prediction mode decision in H.264/AVC

Fast intra prediction mode decision is one of the most important topics in H.264/AVC. There have been three types for fast intra prediction mode decision. These types are based on mode features, block features, and edge or directional information. In this paper, we propose an algorithm for each type. Through extensive simulations, we show that our proposed algorithm using an unconditional DC mode for the type based on mode features significantly saves up to 82.13% of the entire encoding time compared to H.264 reference software while reducing negligible peak signal-to-noise ratio (PSNR) values and slightly increasing bitrate. Another proposed algorithm using a ratio of variance along the horizontal direction to variance along the vertical direction for the type based on edge or directional information keeps a bitrate increment as 0.45% on average while reducing entire encoding time and slightly increasing PSNR values.