Ontology evaluation: Which test to use

There are various methodologies to evaluate ontologies. One of them is based on the percentage coverage of ontology for domain knowledge i.e. gold samples. This paper demonstrates several experiments to show how we can use this ontological coverage to test the quality of ontology. The main concern here is how to use the coverage measure to accept or reject the quality of ontology. Two domains ontologies for Java and .Net have been chosen. Two ontologies in each domain with five frequencies have been selected. Ten gold definitions have been selected in each domain. The coverage has been computed for each definition. The coverage measure has been normalized by divining it on the number of words in the definitions (definition normalization). The second ontology in both domains has been chosen to be better than the first one by including the first ontology in the second one with more concepts in the second. The percentage coverage measures have been computed for the four ontologies with 50 (10*5 frequencies) cases. Finding the “good” population between two populations is a well-know statistical problem. Four techniques have been borrowed: 1) Non-parametric statistical method Mann-Whitney-Wilcoxon U (MWW) test, 2) The average (mean analysis) of the coverage, 3) The average differences in coverage, and 4) The percentage of positive differences in coverage between any two ontologies (sign test). Results show that the technique number 4 (sign test) gives significant results among other tests.