Adaptive Pipeline Parallelism for Image Feature Extraction Algorithms

Currently, multimedia data has become one of the major data types processed and transferred on the Internet. With the rapid growth of multimedia data, it is vitally important to find an efficient way to extract useful information from a large amount of data. SIFT and SURF, as the most popular multimedia feature extraction algorithms, have been widely used in many applications. However, the limited processing speed~(about 1.8 and 2.6 images or frames per second for SIFT and SURF respectively on an ordinary CPU) makes it impossible to apply them in many real-world applications with real-time requirements. Therefore, it has become one of the major challenges that how to improve the processing speed of these multimedia feature extraction algorithms. The popularity of multi-core architecture and the increase of computation resources on different platforms provide a new opportunity to accelerate the processing speed of these image feature extraction algorithms. In this paper, we first systematically analyze the major parallel constraints in SIFT and SURF, such as imbalanced workload and indeterminate time distribution. Then, based on these analysis, we design and implement an adaptive pipeline parallel scheme (AD-PIPE) for both SIFT and SURF to alleviate these limitations. In our scheme, we dynamically check the workload in different pipeline stages and adjust the thread number in different stages to achieve a balanced partition. Experimental results show that our approach is efficient and scalable. It can achieve a speedup of 16.88X and 20.33X respectively for SIFT and SURF on a 16-core machine and a real time processing speed with about 27 and 52 images or frames per second.