Scene-space Feature Detectors

Derivative, spectral, and other operators implemented in filter kernels and applied with convolution have been well studied in image recognition problems. We explore in this paper a novel system that applies such filters in the actual 3D scene. The system projects images of filters into the scene as controlled illumination. The projected filters are subjected to object surface reflectance (BRDF) and scene composition; and the filter response is captured by camera pixels. Our approach utilizes a pixel-based convolution formulation, exploiting the intrinsic integration performed by individual pixel sensors and diffraction-limited optics. For example, we project a grid of Gaussian derivative filters as images and simply retrieve a filter response as a pixel value. This is a local (pixel-only) computation that is not limited by camera resolution. Our system can serve as a front-end to existing techniques for surface or geometry estimation from texture analysis. A projector and camera pair is the basic equipment requirement.