A framework for recovering affine transforms using points, lines or image brightnesses

Image deformations due to the relative motion between an observer and an object may be used to infer 3D structure. Up to first order, these deformations can be written in terms of an affine transform. A new framework for measuring affine transforms, which correctly handles the problem of corresponding deformed patches, is presented. In this framework, points, lines or image brightnesses may be used to derive the affine transform between image patches. No correspondence is required. The patches are filtered using Gaussians and derivatives of Gaussians, and the filters are deformed according to the affine transform. The problem of finding the affine transform is therefore reduced to that of finding the appropriate deformed filter to use. The method is local and can handle large affine deformations. Experiments demonstrate that this technique can find scale changes and optical flow in situations where other methods fail