A hybrid representation of environment models in image-based real time walkthrough

In this paper, a hybrid representation of environment models using a combination of points and polygons is proposed. Through the model, a real time walkthrough of a complex environment can be achieved. We start from multiple depth reference images and classify the pixels of images into two categories, corresponding respectively to planar and non-planar surfaces in 3D space, then all the redundant points are removed by a comparison algorithm of sampling rate. For pixels corresponding to nonplanar surfaces, the point representation is maintained, and a local reconstruction and resampling process is employed to re-sample a set of points that distribute more uniformly on the surfaces. The re-sampled points are organized by an OBB-tree according to their space coordinates and a multiresolution structure is built to improve the rendering efficiency. For pixels corresponding to planar surfaces, the plane coefficients are reconstructed and all their appearances in the reference images are merged to texture maps. Under this hybrid representation, texture mapping and point-based rendering are employed to render the novel views, to take full advantage of the acceleration utility of graphics hardware. The algorithm demonstrates the combined advantages of approaches in traditional computer graphics and PBR/IBR, texture mapping for plane surfaces and point-based rendering for high detail surfaces and shapes.