Frame Rate Fusion and Upsampling of EO/LIDAR Data for Multiple Platforms

We propose a method for fusing a LIDAR point cloud to camera data in real time, which will also backfill the myriad of data holes LIDAR creates. This is done in a way that also leverages the images features to weight how point clouds are filled. Multithreaded programing and GP-GPU methods allow us to obtain 10 fps with a Velodyne 64E LIDAR completely fused in 360^o using a Ladybug panoramic camera. The method also generalizes to other kinds of point clouds such as those obtained by aerial vehicles. The primary advantage of our approach is it combines 360^o fusion with upsampling in real time without mode smoothing.