Resolving occlusion in multiframe reconstruction of deformable surfaces

Occlusion is troublesome for almost all computer vision algorithms. To a certain extent, the difficulty is alleviated when multiple frames are given. On the other hand, when we consider the recovery of shapes of moving deformable objects, observed using a monocular camera, the problem appears difficult again. In this paper, we show a method that outperforms previous approaches to reconstruction when feature data is unavailable, perhaps due to occlusion. Our key intuition is that portions of the surface that are visible in some frame can be reliably reconstructed in that frame; further, the reliable portions can be stitched together to find even missing portions, much the way a human eye would hallucinate. Our techniques are based on optimization in Riemannian shape spaces, and is demonstrated on isometric surfaces without involving any kind of machine learning methods.