Title :
Morphable 3D models from video
Author_Institution :
Mitsubishi Electr. Res. Labs, Cambridge, MA, USA
Abstract :
Nonrigid 3D structure-from-motion and 2D optical flow can both be formulated as tensor factorization problems. The two problems can be made equivalent through a noisy affine transform, yielding a combined nonrigid structure-from-intensities problem that we solve via structured matrix decompositions. Often the preconditions for this factorization are violated by image noise and deficiencies of the data visa-vis the sample complexity of the problem. Both issues are remediated with careful use of rank constraints, norm constraints, and integration over uncertainty in the intensity values, yielding novel solutions for SVD under uncertainty, factorization under uncertainty, nonrigid factorization, and subspace optical flow. The resulting integrated algorithm can track and reconstruct in 3D nonrigid surfaces having very little texture, for example the smooth parts of the face. Working with low-resolution low-texture "found video," these methods produce good tracking and 3D reconstruction results where prior algorithms fail.
Keywords :
image morphing; image motion analysis; image reconstruction; image sequences; stereo image processing; video signal processing; 2D optical flow; image noise; low-resolution low-texture video; morphable 3D models; noisy affine transform; nonrigid 3D structure-from-motion; nonrigid structure-from-intensities problem; norm constraints; rank constraints; sample complexity; structured matrix decompositions; subspace optical flow; tensor factorization problems; tracking; uncertainty; Image motion analysis; Image reconstruction; Integrated optics; Matrix decomposition; Optical noise; Subspace constraints; Surface reconstruction; Surface texture; Tensile stress; Uncertainty;
Conference_Titel :
Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on
Print_ISBN :
0-7695-1272-0
DOI :
10.1109/CVPR.2001.990997