Delta Voxel Cone Tracing

Mixed reality applications which must provide visual coherence between synthetic and real objects need relighting solutions for both: synthetic objects have to match lighting conditions of their real counterparts, while real surfaces need to account for the change in illumination introduced by the presence of an additional synthetic object. In this paper we present a novel relighting solution called Delta Voxel Cone Tracing to compute both direct shadows and first bounce mutual indirect illumination. We introduce a voxelized, pre-filtered representation of the combined real and synthetic surfaces together with the extracted illumination difference due to the augmentation. In a final gathering step this representation is cone-traced and superimposed onto both types of surfaces, adding additional light from indirect bounces and synthetic shadows from anti-radiance present in the volume. The algorithm computes results at interactive rates, is temporally coherent and to our knowledge provides the first real-time rasterizer solution for mutual diffuse, glossy and perfect specular indirect reflections between synthetic and real surfaces in mixed reality.