3D keypoint detection by light field scale-depth space analysis

We present a method for 3D keypoint detection from light field data, typically obtained by planar camera arrays or plenoptic cameras. The proposed approach is based on construction of novel light field scale-depth spaces that are designed to leverage the specific properties of light fields. The constructed scale-depth spaces are based on a modified Gaussian kernel that is parametrized both in terms of scale of objects recorded by the light field and in terms of objects´ depth. We prove theoretically that the new scale-depth space formulation and its spatial derivative satisfy the scale invariance property for all depths. By finding local extrema in such scale-depth spaces we locate 3D keypoints (such as 3D edges) and show that they outperform SURF keypoints on a 3D structure estimation task, both in accuracy and computational efficiency.