Motion-based object segmentation using frame alignment and consensus filtering

Segmentation of moving objects in video sequences has many applications such as video surveillance, traffic monitoring, and object-based video coding. In this work, we propose a novel algorithm that separates locally moving objects (foreground) in a video from a globally moving background using both temporal and spatial contexts. The algorithm consists of four stages: frame alignment, pixel alignment, consensus filtering, and spatio-temporal refinement. The primary contributions of our work are the use of a neighborhood-based similarity metric for pixel alignment and a simple yet efficient spatio-temporal refinement method. Results are shown to be better than those of state-of-art median filtering-based segmentation algorithms.