Practical display-camera calibration from eye reflections using coded illumination

Display-camera systems enable a large range of vision applications in everyday environments, but require calibrating the pose of the display with respect to the camera. Recently, a promising approach is introduced that analyzes corneal reflections from face images. While having several benefits, the described implementation requires controlled and unnatural conditions, manual elaboration, and offline processing. This work aims to improve the approach and allow for conditions of practice: First, the novel idea is proposed to encode display correspondences into illumination patterns to enable automatic and robust detection. This comprises the first use of coded illumination with corneal reflection analysis. Second, since previous evaluation revealed a large error for the basic algorithm, this work discusses thorough experimental evaluation of an optimization strategy based on geometry constraints in the scene. Results show that considerable improvement can be achieved under common parameter variation, which successfully verifies the feasibility of the approach.