3D-Ferns+: Viewpoint-based keypoint classifier for robust 3D object pose detection

We present a novel pose detection method that can be used in mobile augmented reality (AR) services. Making 3D object pose detection robust against changes in viewpoint is a vitally important but quite difficult task because 3D objects often change their appearance significantly with changes in viewpoint, and the possible range of viewpoints is wide compared with planar targets. 3D-Ferns, which is a keypoint classifier for 3D object pose detection, performs direct 2D-3D matching and handles a wide range of detectable viewpoints, including all rotations. However, many difficult viewpoints still exist for pose detection because of the unevenness of matching performance over all viewpoints. In this paper, we propose a novel class selection strategy that evens out matching performance over all possible viewpoints and improves detection performance from difficult viewpoints by focusing on the per-viewpoint repeatability (PVR) of class 3D points. Experimental results demonstrate the impact of stability of 2D-3D matching on detection performance and the effect of our method, which reduces the detection failures in conventional approaches by over 23% for 3D targets that have various shapes and textures.