Improving Segmentation Boundaries with Nonparametric Image Parsing

Semantic segmentation, or segmenting all the objects in an image is one of the core problems of computer vision. In order to achieve an object-level semantic segmentation, we propose to label image regions and to improve the segmentation result based on these labels. We build upon the recent super parsing approach, which is a nonparametric solution to the image labelling problem. We propose to initialize the segmentation with SLICO super pixels because SLICO is able to produce accurate boundaries and offers control over size, shape and compactness of the super pixels. These super pixels are labelled with super parsing but an optimization step is required for the large number of small super pixels. We formulate a Conditional Random Field (CRF) using a novel pair wise cost depending on local features and computed in a nonparametric estimation. This results in stronger semantic contextual constraints. We evaluate our improvements to the super parsing approach using segmentation evaluation measures as well as the per-pixel rate and average per-class rate in a labelling evaluation. We demonstrate the success of our modified approach on the SIFT Flow dataset.