Learning a Probabilistic Similarity Function for Segmentation

There are many methods for measuring "similarity" of or "distance" between two image pixels or image patches. These methods generally involve computing some informative features of the image patches and then describing distances between patches based on simple functions of those feature values, e.g., Euclidean distance. Unfortunately, these measures are often frail and difficult to interpret. The goal of this paper is to learn the "similarity" of two patches as an approximation to the likelihood that two patches were drawn from the same surface in the world. This measure is well-defined and allows for a maximization of meaningful values when it is combined with common segmentation algorithms. We introduce a general approximation technique that involves learning a codebook, learning the likelihood function for pairs of codebook entries, and applying the resulting likelihood function in segmentation tasks. These steps can be performed independently, even on different data sets drawn from the same domain. The likelihood can be learned from pre-segmented image set or heuristically approximated from an unsegmented image set. We show examples of probabilistic segmentation of the codebooks themselves based on our similarity measure for multiple types of codebooks including color codebooks and Epitome codebooks. These segmentations illustrate the usefulness of our technique as an image patch similarity measure.