“Bag of keypoints”-based biomedical image search with affine covariant region detection and correlation-enhanced similarity matching

This paper presents a “bag of keypoints” based biomed-ical image retrieval approach by detecting affine covariant regions. These regions refers to a set of pixels or interest points which are invariant to affine transformations, as well as occlusion, lighting and intra-class variations. The interest points are described with the Scale-Invariant Feature Transform (SIFT) and vector quantized to build a visual vocabulary of keypoints. By mapping the interest points extracted from one image to the keypoints in the vocabulary, their occurrences are counted and the resulting histogram is called the “bag of keypoints” for that image similar to the “bag of words” based representation of documents in text retrieval. To exploit the correlations between the keypoints in the collection, a global similarity matrix is constructed to be utilized in a distance measure function to compare the query and database images. A systematic evaluation retrieval results on a biomedical image collection demonstrates around 10-15% improvement in precision at different recall levels for the keypoint-based representation along with the correlation-enhanced distance measure when compared to individual color, texture, edge-related features.