Semantic variance: An intuitive measure for ontology accuracy evaluation

Ontology evaluation is a relevant issue in the field of knowledge representation. It aims at quantifying the quality of ontologies, so that potential users can have an idea of their accuracy and thereby select the most appropriate ontology for a specific application. Many of the ontology evaluation methods and frameworks available in the literature assess the quality of ontologies according to their structural features, even though most of these methods propose ad hoc aggregations of such features that lack a theoretical basis. Inspired by recent empirical studies showing that some structural features are better suited to predict the semantic accuracy of ontologies, we present in this paper the notion of semantic variance of an ontology. Semantic variance is an intuitive and inherently semantic measure to evaluate the accuracy of ontologies. Unlike ad hoc methods, our proposal is a mathematically coherent extension of the standard numerical variance to measure the semantic dispersion of the taxonomic structure of ontologies. In our experiments performed over a set of widely used ontologies, the proposed semantic variance positively correlated with the structural features of ontologies that best predicted their accuracy in previous studies. Moreover, our measure also provided a good prediction of the ontological accuracy in one of the most essential knowledge-based tasks: assessing the semantic similarity between concepts. These results suggest that the semantic variance can be used as a generic, quantitative and theoretically coherent score to evaluate the accuracy of ontologies.