A Large Scale Concept Ontology for News Stories: Empirical Methods, Analysis, and Improvements

We analyze the completeness, accuracy, and utility of the largest known annotation ground truth database for video news stories, comprising nearly 680K individual tags on 62 K shots using a vocabulary of 449 semantic concepts. We find the vocabulary is not yet mature: it does not follow Zipf\´s law, although concepts derived from vocabulary intersection do so more closely. We find that because many concepts are sparse, the best method for exposing the implicit semantic space is to use the distance measure G², complete link clustering, and a heuristic distance cutoff based on shot cluster evolution history; this yields 12 well-defined major shot categories. Because the database is errorful, we derive a model for annotator error, and using it, we extract a natural concept subsumption ontology from the database, including some counter-intuitive relationships. Again using intersection, we demonstrate a method for identifying "missing" subconcepts from superconcepts. Lastly, we note that without superconcepts, shot clustering fails. These methods are unbiased by any prior semantic assumptions, and only depend on a statistically sufficient body of ground truth. They are therefore applicable to any other specific video retrieval domain.