Challenges in Segmentation of Human Forms in Outdoor Video

Most gait and activity recognition algorithms rely on the use of silhouettes as the low-level representation. However, the detection of good silhouettes is still an open problem, particularly for sequences that are taken outdoors. Illumination conditions, compression artifacts, and low number of pixels on the subject, contribute to the difficulty. Presently, these issues are either ignored by using indoor data or addressed, on a case by case basis, by employing, essentially, a "bag of tricks" based approach. We argue for a more formal approach, based on generic shape and motion models to handle a variety of these problems, under the umbrella of one formalism. We present an HMM-based Eigen Stance model, built based on manually created silhouettes from 71 individuals. The population HMM helps map a frame in any given sequence to a stance and the appearance based Eigen-Stance model is used to reconstruct the computed silhouette in that frame. We quantify the performance in terms of signal based criteria of missed detection and false positive prediction rate. We also show results on three different databases.