Improved specular regions localization and optical-flow based motion estimation via joint processing

For specular regions (SRs), the assumption of brightness (or other) constancy between images corresponding to multiple views of a scene breaks down. As a consequence, optical-flow (OF) based motion-estimation (ME) algorithms that rely on constancy assumptions fail for specular regions. At the same time estimation of SRs in an image is also prone to errors, particularly to false positives from bright regions in the scene. In this paper, motivated by the fact that specular regions are typically encountered in image regions corresponding to portions of relatively smooth 3D surfaces, we propose an algorithm for improving ME and SRs localization via joint processing. Initial estimates of OF and of the SRs are obtained by conventional methods. The estimate of the SRs is updated using inconsistency of the OF with respect to the neighboring region to reinforce true positives and to reject false positives. The OF is then re-computed with a modified energy functional that, in effect, emphasizes regularization in a spatially adaptive neighborhood of the SRs to improve the estimated OF. Experimental results on synthetic and real image pairs demonstrate that the proposed algorithm offers a significant improvement in both SRs localization and ME over recently proposed methods for tackling these problems.