Content-based vehicle retrieval using 3D model and part information

Content-based vehicle retrieval in unconstrained environment plays an important role in surveillance system. However, due to large variations in viewing angle/position, illumination, and background, traditional vehicle retrieval is extremely challenging. We approach this problem in a different way by rectifying vehicles from disparate views into the same reference view and searching the vehicles based on informative parts such as grille, lamp, and wheel. To extract those parts, we fit 3D vehicle models to a 2D image using active shape model (ASM). In the experiments, we compare different 3D model fitting approaches and verify that the impact of part rectification on the content-based vehicle retrieval performance is significant. We propose a model fitting approach with weighted jacobian system which leverages the prior knowledge of part information and shows better results. We compute mean average precision of vehicle retrieval with L1 distance on NetCarShow300 dataset, a new challenging dataset we construct. We conclude that it benefits more from the fusion of informative rectified parts (e.g., grille, lamp, wheel) than a whole vehicle image described by SIFT feature for content-based vehicle retrieval.