Mixing retrieval and tracking using compact visual descriptors

Visual search has seen many improvements over the years, but its application on video content is still an open research problem and it is often limited to still images. Based on the tools devised by the standardization group MPEG CDVS, we developed a processing flow that processes at nearly real-time a video acquired with a low cost imager and performs content search and retrieval of the top match from a local database. To allow efficient interest point detection, we used a GPU accelerated SIFT library. To process the video frames efficiently, we developed a new dataflow processing which allows switching between object searching, retrieving and tracking in order to keep at minimum the number of queries sent to the database. A search into a local database is performed only when no object has been recognized, and once a good match has been found, the algorithm switches to tracking mode.