A Depth-Dependent Fusion Algorithm for Enhanced Reality Based on Binocular Vision

As one of the pivot techniques in enhanced reality the fusion of virtual and real scene image often happens in two ways, either to incorporate a real object into a virtual environment, or to incorporate a virtual object into a real scene. Most of the conventional methods for the fusion are conducted at the image level, which fails to handle three-dimensional objects. In this paper, a novel depth-dependent algorithm for the fusion of virtual and real scene image in enhanced reality based on binocular vision will be proposed. The algorithm simulates the human observation process, and computes the spatial position of an image pixel by means of binocular parallax. With the assistance of object recognition techniques, the depth computation results will then be utilized to determine the spatial position of the object in the observation coordinate frame. The fusion is consequently implemented by positioning the object of interest at the right position and establishing the depth relationship with other objects in the scene. Experiment results demonstrate that the proposed algorithm is both valid and effective to achieve a high quality fusion for an enhanced reality application.