Recovering shape and irradiance maps from rich dense texton fields

We describe a method that recovers an estimate of surface shape and of the irradiance field for a textured surface. The method assumes the surface is viewed in scaled orthography, and we demonstrate the appropriateness of this assumption. Our method uses interest points to obtain the locations of putative texton instances, clusters the textons into types, and then uses an autocalibration method to recover the frontal appearance of each texton model. This yields (a) a dense set of normal estimates, each up to a two-fold ambiguity, (b) a dense set of irradiance estimates and (c) whether each instance is, in fact, an instance of the relevant texton. Because we are able to obtain a very large number of instances of a large number of different textons, this information is obtained at sites very closely spaced in the image. As a result, we need only a simple smoothness constraint to reconstruct a surface model, using EM to resolve the normal ambiguity. We show results on images of real scenes, comparing our reconstructions with those obtained using other methods and demonstrating the accuracy of both the recovered shape and the irradiance estimate.