Three dimensional high dynamic range veillance for 3D range-sensing cameras

This paper presents the invention and implementation of 3D (Three Dimensional) HDR (High Dynamic Range) sensing, along with examples. We propose a method of 3D HDR veillance (sensing, computer vision, video capture, or the like) by integrating tonal and spatial information obtained from multiple HDR exposures for use in conjunction with one or more 3D cameras. In one embodiment, we construct a 3D HDR camera from multiple 3D cameras such as Kinect sensors. In this embodiment the 3D cameras are arranged in a fixed array, such that the geometric relationships between them remain constant over time. Only a single camera calibration step is required at the initial time of assembling and fixing the cameras into the array. Preferably the cameras either view from the same position through beam splitters, or are fixed close to one another, so that they capture approximately the same subject matter. The system is designed so the cameras each capture a differently exposed image or video of approximately the same subject matter. In one embodiment, two Kinect cameras are attached together facing in the same direction, with an ND (Neutral Density) filter over one of them, so as to obtain a darker exposure. The dark and light exposures are combined to obtain more accurate 3D sensing in high contrast scenes. In another embodiment, a single 3D camera is exposure-sequenced (alternating light and dark exposures). 3D HDR might, more generally, be incorporated into existing 3D cameras, resulting in a new kind of 3D sensor that can work in nearly any environment, including high contrast scenes such as outdoor scenes, or scenes where a bright light is shining directly into the sensor.