On-Device Mobile Visual Location Recognition by Integrating Vision and Inertial Sensors

This paper deals with the problem of city scale on-device mobile visual location recognition by fusing the inertial sensors and computer vision techniques. The main contributions are as follows: Firstly, we design an efficient vector quantization strategy by combining the Transform Coding (TC) and Residual Vector Quantization (RVQ). Our method can compress a visual descriptor into only several bytes while providing reasonable searching accuracy, which makes the managing of city scale image database directly on mobile devices come true. Secondly, we integrate the information from inertial sensors into the Vector of Locally Aggregated Descriptors (VLAD) generation and image similarity evaluation processes. Our method is not only fast enough for on-device implementation, but it also can improve the location recognition accuracy obviously. Thirdly, we also release a set of 1.295 million geo-tagged street view images with the information from inertial sensors, as well as a difficult set of query images. These resources can be used as a new benchmark to facilitate further research in the area. Experimental results prove the validity of the proposed methods for on-device mobile visual location recognition applications.