A differential method for computing local shape-from-texture for planar and curved surfaces

We model the texture distortion at a point in any particular direction on the image plane as an affine transformation and derive the relationship between the parameters of the affine transformation and the surface shape and orientation. We use a technique for estimating affine transforms between nearby image patches which is based on solving a system of linear constraints derived from a differential analysis. It is not necessary to explicitly identify texels or make restrictive assumptions about the nature of the image texture like isotropy. We have developed two different algorithms for recovering surface orientation and shape based on the estimated affine transforms in a number of different directions. The first is a sample linear algorithm based on singular value decomposition. The second is based on nonlinear minimization of a least squares error criterion. Experimental results are presented on images of planar and curved surfaces under perspective projection