Eye gaze based reading detection

We propose a system to determine whether a computer user is engaged in reading activity on the computer monitor, or not, using sequences acquired from an ordinary video camera. The challenge is to differentiate reading from other activities that the user can perform at the computer and accommodate user differences. The proposed method is based on estimating gaze directions, followed by determination of reading. A set of finite state machines (FSMs) are used to represent reading by modeling the changes in gaze direction that take place during reading. To determine gaze direction, the face is first located. The eyes are then found within the face and tracked over successive frames. Gaze directions are then estimated from the tracked eyes over a finite time interval (window) and checked with respect to the FSMs to determine whether the interval represents reading activity. An aggregate of these determinations over successive time windows is finally used to infer whether reading is taking place. An average reading detection of 85.4% was achieved on 10 people with a false alarm rate of 14.2%; however, the system at present is user dependent, and some components of the system must be trained for individual users.