Uncalibrated non-rigid factorisation with automatic shape basis selection

We propose an extension to the non-rigid factorisation method to solve the affine structure and motion of a deformable object, where the shape basis is selected automatically. In contrast to earlier approaches, we assume a general uncalibrated, affine camera model whereas most of the previous approaches assume a special case such as an orthographic, weak-perspective or paraperspective camera model. In general, there is a global affine ambiguity for the shape bases. It turns out that a natural way of selecting the shape bases is to pick up the bases that are statistically as independent as possible. The independent bases can be found by independent subspace analysis (ISA) which leads to the minimisation of mutual information between the basis shapes. After selecting the shape basis by ISA, only the in-the-subspace affine ambiguities remain from the general affine ambiguity. To solve the remaining unknowns of the general affine transformation, we propose an iterative method that recovers the block structure of the factored motion matrix. The experiments are provided with synthetic structure and real face expression data in 2D and 3D, which show promising results.