Improving KLT in Embedded Systems by Processing Oversampling Video Sequence in Real-Time

We propose an efficient hardware architecture for Kanade-Lucas-Tomasi(KLT) algorithm to process over-sampling video data in real-time. The philosophy of this paper is to make sure the tracking performance of KLT by decreasing the inter-frame displacement rather than handling large displacement by complex algorithms. It would be a preferable method for embedded systems because of their higher special-purpose instead of general-purpose computational performance. According to this architecture, the time to extract all features is just the time to capture one frame of image. This time interval can be used to track as many features as possible. This architecture was implemented in a Xilinx Virtex-5 FPGA. As shown in the synthesized result, the maximum frequency of the system can be up to 182.287MHz. Thus, around 182 frames of image with 1024 × 768 resolution can be processed and more than 1000 features can be tracked successfully within each frame. This makes the inter-frame displacement really small in contrast to that of the lower frame rate provided before. Empirical analysis shows that when the displacement small enough(e.g., no more than 5-pixel), features can be tracked well based on just the basic KLT algorithm.