Scene understanding from propagation and consistency of polarization-based constraints

We show how the more general capabilities of polarization vision can be used to significantly enhance the understanding of imaged scenes particularly at regions containing specular and mirror reflections, and occluding contours. Intensity-based methods are easily confounded by specular reflection. Specular mirror reflection of an object is indistinguishable from the object itself when using intensity and color, but is shown to be distinguishable when reflected polarization is analyzed. Polarization-based methods also enhance the identification of occluding contour edges, and regions that are in the vicinity of occluding contour edges. In past work local polarization constraints have been shown to accurately label specularities and occluding contours in simple scenes. In more complicated scenes under general conditions local polarization constraints are usually not sufficient to produce an accurate labeling of these phenomena. The main new idea presented in this paper is to extract much more additional information by enforcing physical consistency between neighboring local polarization constraints and propagating these constraints across pixel edges and regions with significant partial polarization. Initial labeling of pixels is produced from local polarization constraints at edges. We present well-defined physically motivated rules for relabeling pixels where physical inconsistencies result between neighboring local polarization constraints. Propagation rules allow physically consistent labeling of regions of significant partial polarization, and a refinement rule allows some more detailed labeling of edges. We include the previously unstudied case of specular reflection at/near occluding contours. In all 10 edge and region labels are generated. Experimental results on a complex scene is presented demonstrating the vast amount of information that can be extracted from these new physical principles of polarization vision