A Markov Random Field model for decision level fusion of multi-source image segments

We present a method based on Markov Random Fields (MRFs) for conducting decision level fusion of segments derived from multiple images of the same region. These images are not required to share the same resolution or sensor characteristics. By working at the segment level we preserve the advantages of segment based image classification while also incorporating the benefits of using multiple image sources. Segment fusion is achieved by constructing a MRF graph over the segments with an edges connecting overlapping segments from different images. These edges penalize connected segments for taking different labels as a function of the degree of overlap. Experimentation on the fusion of Land-sat and SPOT5 imagery for classification of different forest types shows the ability of this method to deliver substantial improvements in classification accuracy.