Shadow elimination for robust video surveillance

Variation in illumination conditions, caused by weather, season, time of day, etc., makes the task difficult when building surveillance systems of real world scenes. Especially, cast shadows produce troublesome appearance variations to accomplish monitoring with computer vision techniques, typically moving object tracking from a stationary viewpoint. To eliminate lighting effects robustly from image sequences as a preprocessing stage for robust video surveillance, we propose a framework based on the idea of intrinsic images. Unlike previous methods to derive intrinsic images, we derive time-varying reflectance images. As a result, we obtain illumination images that capture only lighting effects on the scene. Using these reflectance images and illumination images, we construct an illumination eigenspace to estimate directly the illumination images under the arbitrary lighting conditions of the same scene. By canceling out the lighting effects using this illumination image, robust video surveillance can be accomplished. We explain the theory of the framework with simulation results, and apply the framework to a real world monitoring data set to prove its effectiveness.