Soccer Video Annotation Using Ontologies Extended with Visual Prototypes

This paper presents multimedia ontologies, where visual prototypes are added as specialization of concepts defined in a traditional linguistic ontology. The more general concepts of the sport domain (e.g. play/break, crowd, etc.) are put in correspondence with the more general visual features of the video like color and texture, while the more specific concepts of the soccer domain (e.g. highlights such as attack actions) are put in correspondence with domain specific visual feature like the soccer playfield and the players. A solution for the implementation of multimedia ontologies for the soccer video domain and a method to perform automatic soccer video annotation using these extended ontologies is shown. Experimental results for automatic annotation of soccer videos containing generic concepts are presented.