A hardware implementation of pyramidal KLT feature tracker for driving assistance systems

In this paper, we describe an FPGA based implementation of pyramidal KLT (Kanade_Lucas_Tomasi) feature tracker that can be applied to vision based driving assistance systems where real-time operation is commonly required. The implementation consists of three functional modules: feature extraction, feature tracking, and reselection of lost features. The description mainly focuses on alleviation of the following problems which are common places in this type of applications: 1) loss of feature points due to diverse illumination conditions, 2) computational complexity involved in sorting extracted features and reselection procedures, and 3) excessive number of assesses to sub-images in memory while pyramids are constructed and features are being tracked. In our hardware implementation, a histogram based intensity adaptive thresholding scheme is introduced for the feature extraction and the reselection. In addition, a double buffering scheme and inherent parallelism of FPGA are fully exploited for efficient construction of pyramids and feature tracking. The experimental results show that the tracker is capable of handling input streams at 60 fps without skipping a single frame.