Non-calibrated and real-time human view estimation using a mobile corneal imaging camera

We present a mobile human view estimation system using a corneal imaging technique. Compared to the current eye gaze tracking (EGT) systems, our system does not require per-session calibrations and a frontal view (scene) camera, making it suitable for wearable glass systems because it is easier to use and more socially acceptable due to the lack of a frontal scene camera. Our glasses system consists of a glass frame and a micro eye camera that captures the eye (corneal) reflections of a user. 3D corneal pose tracking is performed for the captured images by using a particle filter-based real-time tracking method leveraged by a 3D eye model and weak perspective projection. We then compute the gaze reflection point (GRP) where the light from the point of gaze (PoG) is reflected, enabling us to identify where a user is looking in a scene image reflected on the corneal surface. We conducted experiments using a standard computer display setup and several real-world scenes, and found that the proposed method performs with considerable accuracy under non-calibrated setups. This demonstrates its potential for various purposes such as the user interface of a glasses systems and the analysis of human perceptions in actual scenes for marketing, environmental design, and quality-of-life applications.