Registration of an uncalibrated image pair to a 3D surface model

The following data fusion problem is considered: Given a 3D geometric model of an object and two uncalibrated images of the same object, and assuming that the object surface is textured and Lambertian, precisely register the images to the model. Solving this problem is necessary for building a geometrically accurate, photorealistic model from laser-scanned 3D data and high quality images. We generalise the photo-consistency approach by Clarkson et al. to the case of uncalibrated cameras, when both intrinsic and extrinsic parameters are unknown. This gives a user the freedom of taking the pictures by a conventional digital camera, from arbitrary positions and with varying zoom. We discuss a number of possible approaches to the problem and propose a method based on manual preregistration followed by a genetic optimisation algorithm. The issues of speed and robustness are addressed. Results for real data are shown.