Autonomous learning of visual concept models

As the amount of video data increases, organizing and retrieving video data based on their semantics is becoming increasingly important. Traditionally, supervised learning is used to build models for detecting semantic concepts. However, in order to obtain a substantial amount of training data, extensive labeling work is needed with the supervised learning schemes. In this paper, we propose a novel autonomous learning framework in which imperfect labelling automatically extracted from cross-modality information is used for training. This completely avoids the manual labeling process. In our proposed framework, imperfect labels without user involvement are first obtained from cross-modality information. Then, based on our proposed new schemes, "generalized multiple-instance learning" and "uncertain labeling density", the system conjectures relevance scores of visual concepts. From these scores, support vector regression is used to build generic visual models. In preliminary experiments, we use the proposed system to learn 20 visual concepts in 6 hours of video. Compare with two concept models that were trained by two supervised algorithms, this novel autonomous learning framework achieves better system average precisions. Other concept models also show promising results.