Semantic Retrieval with Enhanced Matchmaking and Multi-Modality Ontology

This paper describes a semantic retrieval system that allows matchmaking with ranked output and the use of multi-modality ontology to retrieve animal images. Our multi-modality ontology, which integrates both image features and text information, is extended to provide a ranking mechanism. Ranking is calculated from correlation in each modality and is used to refine the semantic matchmaking result. To benchmark our results, we use the top 200 images of Google Image Search for each category to do the experimental comparison. Google Image Search claims to be the most comprehensive on the Web, with billions of images indexed and available for viewing. For different categories of animals in the canine family, we found averages of about 60% of top 200 images are correct images. Google returns even more false results outside this range. Therefore any bigger image set will become meaningless in our experiment. The medium size of the data set is fine since we are testing the retrieval performance on web images and concerned mainly with the precision of top retrievals. We believe the canine domain is challenging as demonstrated by the visual variance of objects and backgrounds. Twenty animal categories, containing animal images and corresponding web pages, are collected to form a systematic animal family. Results show that we can classify perceptually close animal species which share similar appearances as we can infer their hidden relationships from the canine family graph. By assigning a ranking to the semantic relationships we show unequivocal evidence that our improved model achieves good accuracy.