Neural Network Based Foreground Segmentation with an Application to Multi-Sensor 3D Modeling

This paper presents a technique for foreground/background segmentation using either color images or a combination of color and range images. In the case of images captured from a single 2D camera, a hybrid experience-based foreground segmentation technique is developed using a neural network and graph cut paradigm. This gives an advantage over methods that are based on color distribution or gradient information if the foreground/background color distributions are not well separated or the boundary is not clear. The system can segment images more effectively than the latest technology of graph cut, even if the foreground is very similar to the background. It also shows how to use the method for multi-sensor based 3D modeling by segmenting the foreground of each viewpoint in order to generate 3D models