Eye-contact visual communication with virtual view synthesis

In this paper, we propose a new visual communication system where eye contact is possible by using a virtual image. The virtual image is obtained by view synthesis with stereo matching from two real camera views. We developed a region based dynamic programming (DP) approach with improved matching cost, occlusion cost and vertical smoothness constraint. We also proposed a fast view interpolation method. To achieve real time performance, we developed a hardware system. Furthermore, to avoid the reordering problem in the foreground region, a view change approach with disorder detection is adopted. Experimental results demonstrate the validity of our improved DP matching algorithm and eye-contact visual communication system.