GPU accelerated view synthesis from multiple RGB-D images

This paper proposes a GPU-based approach to generate novel views from a set of RGB-D images captured by static multiple depth cameras. The proposed method consists mainly of two steps: (1) construction of 3D structures by non-rigid registration and (2) an image-based rendering procedure. For seamless registration, we apply a thin plate spline-based deformation onto a 3D point cloud rather than a conventional rigid-body transformation which would result in inaccurate registration due to the low precision of depth measurements from a distance. For rendering, our approach first draws a non-rigidly registered point cloud onto a depth buffer in the GPU and then fills holes and removes noise. Finally, we project an RGB color image onto reconstructed 3D points using the projective texture technique. Since most of the procedures are implemented using programmable CPUs, our methods fit well with modern graphics hardware and therefore can accelerate computationally heavy processes. Experiments showed high-quality seamlessly rendered results of multiple RGB-D images compared with the previous point-based rendering and registration techniques.