View interpolation using epipolar plane images

We present a novel approach for the interpolation of intermediate images from images captured at adjacent viewpoints. The key insight of this approach is that no matter where a camera moves about a scene, for any particular feature, the lines of sight from the camera´s principal point, through that feature in space all intersect at the feature. From mathematical duality, the duals of these lines of sight lie along a feature trajectory whose dual is the scene point. Coupling this duality with several constraints imposed on the image acquisition process, intermediate views can be generated from adjacent views by interpolating lines of sight along the feature trajectories. Pixel-by-pixel correspondence between views is simpler than in a stereo matching system because the densely sampled views change minimally from frame to frame. Visibility determination of holes and overlaps is based on the grouping of segmented regions of epipolar plane images (EPIs) into aspects. Once the aspects are recovered, we use the aspect hierarchy to establish a layered structure of feature trajectories and their visual events