Multi-stage vector quantization towards low bit rate visual search

While much progress has been made in mobile visual search, user experiences still relate to the query transmission latency, especially over a bandwidth-constrained wireless link. Low bit rate visual search paradigm has been well advocated in both academic and industrial endeavors, which directly extracts and sends compact visual descriptor(s) rather than sending a query image. Recent advances in compact descriptor design have advocated the use of compressed bag-of-words histogram, which has shown superior performance over other alternatives. However, existing works focus on descriptor compactness, regardless of time cost and memory requirements on the extraction pipeline, which in turn is crucial for the mobile end development. In this paper, we investigate the problem of designing a memory-light descriptor extraction scheme based upon the so-called multi-stage vector quantization. Our scheme starts by quantizing local patches with a small codebook, and the resulting quantization residual is subsequently compensated by a product quantizer. The design of both quantizers are based upon improving PSNR, which would drop a lot through quantization. PSNR is quantitatively shown to be highly correlated with retrieval and matching accuracy. Extensive evaluation on MPEG Compact Descriptor for Visual Search (CDVS) dataset, has reported superior performance over the state-of-the-art.