Automated registration of 3D-range with 2D-color images: an overview

The automatic creation of photorealistic models of large-scale scenes is an important problem. A new generation of laser range scanners can produce very accurate and dense geometric representations in terms of point clouds of 3D points. One the other hand, images captured by inexpensive regular 2D cameras can populate this geometric representation with a large number of photometric observations. Solutions to the problem of registering these different sources of information thus become crucial. The main difficulty stems from the different acquisition processes: active sensing in 3D and passive sensing in 2D. For example, a normal discontinuity in the 3D world will be captured by both sensors but lighting effects will only be recorded by the 2D camera. This paper provides an overview of the current stat-of-the-art and then summarizes our contributions in the field. There are three categories of solutions to the problem. The first one attacks the problem of registering a single 2D image with the 3D model by matching extracted features from the 2D and 3D space. The second one deals with a textured 3D model and it can thus use 2D-to-2D matching techniques. Finally, the third and more comprehensive approach involves the alignment of a set of 2D images to a 3D model. A number of connections are thus explored (2D-to-2D, 2D-to-3D, and 3D-to-3D) leading to more robust solutions.