Structure from motion using SIFT features and the PH transform with panoramic imagery

Omni-directional sensors are useful in obtaining a 360° field of view of a scene for robot navigation, scene modeling, and telepresence. A method is presented to recover 3D scene structure and camera motion from a sequence of multiple images captured by an omnidirectional catadioptric camera. This 3D model is then used to localize other panoramic images taken in the vicinity. This goal is achieved by tracking the trajectories of SIFT keypoints, and finding the path they travel by utilizing a Hough transform technique modified for panoramic imagery. This technique is applied to spatio-temporal feature extraction in the three-dimensional space of an image sequence, as that scene points trace a horizontal line trajectory relative to the camera. SIFT (scale invariant feature transform) keypoints are distinctive image features which can be identified between images invariant to scale and rotation. Together these methods are applied to reconstruct a three-dimensional model from a sequence of panoramic images, where the panoramic camera was translating in a straight line horizontal path. Only the camera/mirror geometry is known a priori. The camera positions and the world model is determined, up to a scale factor. Experimental results of model building and camera localization using this model are shown.