Embedding Invisible Codes Into Normal Video Projection: Principle, Evaluation, and Applications

We describe a system of embedding codes into projection display for structured light-based sensing, with the purpose of letting the projector serve as both a display device and a 3-D sensor. The challenge is to make the codes imperceptible to human eyes so as not to disrupt the content of the original projection. There is the temporal resolution limit of human vision that one can exploit, by having a higher than necessary frame rate in the projection and stealing some frames for code projection. Yet, there is still the conflict between imperceptibility of the embedded codes and the robustness of code retrieval that has to be addressed. We introduce noise-tolerant schemes to both coding and decoding stages. At the coding end, specifically designed primitive shapes and large Hamming distance are employed to enhance tolerance toward noise. At the decoding end, pretrained primitive shape detectors are used to detect and identify the embedded codes, a task that is difficult to achieve by segmentation that is used in general structured light methods, because the weakly embedded information is generally interfered by substantial noise. Extensive experiments show that the proposed system is effective, even with the prerequisite of incurring minimum disturbance to the original projection.