Refinement of disparity estimates through the fusion of monocular image segmentations

The authors examine how estimates of three-dimensional scene structure, as encoded in a scene disparity map, can be improved by the analysis of the original monocular imagery. They describe the utilization of surface illumination information provided by the segmentation of the monocular image into fine surface patches of nearly homogeneous intensity to remove mismatches generated during stereo matching. These patches are used to guide a statistical analysis of the disparity map based on the assumption that such patches correspond closely with physical surfaces in the scene. Such a technique is quite independent of whether the initial disparity map was generated by automated area-based or feature-based stereo matching. Refinement results on complex urban scenes containing various man-made and natural features are presented, and the improvements due to monocular fusion with a set of different region-based image segmentations are demonstrated