Evaluating Ontologies Based on the Naturalness of Their Preferred Terms

The art and science of building ontologies have been developed to the point where it is not sufficient anymore to design and implement a new ontology. Rather, one needs to follow the process of building an ontology by evaluating its quality in absolute numeric terms. If another ontology in the same domain exists, then the two ontologies should be compared in a quantitative manner to determine which one of them is better. Furthermore, the quality scoring mechanism should provide clues concerning the sections of the ontology (one or both) that need improvement. Ontologies are complex structures which exist in many different variations. Even after imposing a basic structural framework and choosing a domain, two ontologies may be evaluated with respect to a number of different features. In this paper we will concentrate on one single ontology feature and assume that all other features are fixed. We have developed a mechanism to measure the quality of this ontology feature, preferred term(s) based on the concept of naturalness, and show that it agrees very well with human judgments. Thus we provide an approach towards the principled selection of the preferred terms in an ontology.