Vehicle class recognition from video-based on 3D curve probes

A new approach is presented to vehicle-class recognition from a video clip. Two new concepts introduced are: probes consisting of local 3D curve-groups which when projected into video frames are features for recognizing vehicle classes in video clips; and Bayesian recognition based on class probability densities for groups of 3D distances between pairs of 3D probes. The most stable image features for vehicle class recognition appear to be image curves associate with 3D ridges on the vehicle surface. These ridges are mostly those occurring at metal/glass interfaces, two-surface intersections such as back and side, and self-occluding contours such as wheel wells or vehicle-body apparent contours, i.e., silhouettes. There are other detectable surface curves, but most do not provide useful discriminatory features, and many of these are clutter, i.e., due to reflections from the somewhat shiny vehicle surface. Models are built and used for the considerable variability that exists in the features used. A Bayesian recognizer is then used for vehicle class recognition from a sequence of frames. The ultimate goal is a recognizer to deal with essentially all classes of civilian vehicles seen from arbitrary directions, at a broad range of distances and under the broad ranges of lighting ranging from sunny to cloudy. Experiments are run with a small set of classes to prove feasibility. This work uses estimated knowledge of the motion and position of the vehicle. We briefly indicate one way of inferring that information which uses 1D projectivity invariance.