3D pose and camera parameter tracking algorithm based on Lucas-Kanade image alignment algorithm

This paper describes a method to simultaneously estimate 3D pose and camera zoom parameters from sequential images. Given the polyhedral 3D model and its 2D surface texture, 3D pose parameters and camera focal lengths, which yield the best match between the current image and the reference image, are estimated precisely using gradient descent optimization. For the performance evaluation of the proposed algorithm, convergence tests were conducted. 3D pose and camera zoom tracking also conducted on both of synthesized and real sequential images. 3D object information makes the algorithm to effectively cope with self-occlusions, disappearance, and reappearance of partial surfaces of the object by checking visibility for each surface using its 3D pose. As the proposed method estimates camera focal lengths together with 3D rotation and translation, it can be applied to the 3D pose tracking on images of a camera with a zoom lens.