“Clickable real world” information retrieval application based on geo-visual clustering

We propose an intuitive operation based information retrieval system “Clickable Real World (CRW)”. If a user which uses this system takes a picture of a landmark in the world, some related information is displayed on a smartphone. One of the key research issues is how to estimate appropriate keywords in order to retrieve information related to the target. Our strategy utilizes a lot of training samples, which consist of images, tags and geolocation where the image was taken, shared on the Web (such as flickr and Picasa). Then, a keyword table is created based on the consistency of geolocation and visual feature. We developed a prototype version of CRW on a smartphone, and conducted a field experiment in Kyoto city, Japan.