Automatic Alignment of Color Imagery onto 3D Laser Radar Data

We present an algorithm for the automatic fusion of city-sized, 2D color imagery to 3D laser radar imagery collected from distinct airborne platforms at different times. Our approach is to derive pseudo-intensity images from ladar imagery and to align these with color imagery using conventional 2D registration algorithms. To construct a pseudo-intensity image, the algorithm uses the color imagery´s time of day and location to predict shadows in the 3D image, then determines ambient and sun lighting conditions by histogram matching the 3D-derived shadowed and non-shadowed regions to their 2D counterparts. A projection matrix is computed to bring the pseudo- image into 2D image coordinates, resulting in an initial alignment of the imagery to within 200 meters. Finally, the 2D intensity image and 3D generated pseudo-intensity image are registered using a modified normalized correlation algorithm to solve for rotation, translation, scale and lens distortion, resulting in a fused data set that is aligned to within 1 meter. Applications of the presented work include the areas of augmented reality and scene interpretation for persistent surveillance in heavily cluttered and occluded environments.