Conditional random field-based image labelling combining features of pixels, segments and regions

Conditional random field (CRF)-based framework is the most popular approach to image labelling. Pixel-based CRF and segment-based CRF correspond to image representations on different scales. Hierarchical CRF models are the main technique to combine multi-scale information of an image. In this study, the authors propose a single-layered segment-based CRF, instead of multi-layered hierarchical CRF, to integrate multi-scale features of pixels, segments and regions. The unary potential associated with a segment in the CRF is determined by the features of pixels in it, instead of by the statistical features of the segment. On the other hand, the features of a pixel contain features of multiple segments it belongs to. By this means, features of pixels and segments computed at different levels are integrated naturally. Furthermore, to alleviate the problem of local minima and to capture long-range semantic context, the authors propose a region-based CRF to model co-occurrence. Compared with some existing approaches to model co-occurrence, it is relatively fast and can correct some co-occurrence constraints violation errors. Experiments on MSRC-21 database show that our model achieves comparable results to the state-of-the-art algorithms but with lower complexity.