Detection of 3D position of eyes through a consumer RGB-D camera for stereoscopic mixed reality environments

A novel approach to track the 3D position of the user´s eyes in stereoscopic virtual environments, where stereo glasses are worn, is proposed. Such an approach improves a state-of-the-art real-time face tracking algorithm by addressing the occlusion due the stereo glasses and providing estimation of eye position based on biométrie features. More generally, our solution can be seen as a proof of concept for a more robust approach to improving motion tracking techniques. In particular, the proposed technique yields accurate and stable estimates of the 3D position of the user´s eyes, while the user moves in front of the stereoscopic display. The correct tracking of both eyes´ 3D position is a crucial step in order to achieve a more natural human-computer interaction which diminishes visual fatigue. The proposed approach is validated through quantitative tests: (i) we assessed the accuracy of our algorithm for tracking the 3D position of users´ eyes with and without stereo glasses; (ii) we have performed a perceptual assessment of the natural interaction in the virtual environments through experimental sessions with several users.