Improving AR using shadows arising from natural illumination distribution in video sequences

In this paper, we propose a method for generating realistic shadows of virtual objects inserted into a real video sequence. Our aim is to improve and extend the work of Sato, Sato, and Ikeuchi, (1999), which is based on a static camera, to the case of a video sequence. This extension consists of several procedures: calibration of a moving video camera and a graphic camera, removing false shadows occurring due to a shortcoming of the static camera approach for the estimation of an illumination distribution, and so on. The calibration of the moving camera is solved by camera self-calibration and, with it, we designed a flexible graphic world coordinate system embedding technique called “match move”. We also show that the shortcoming of the previous static camera approach is overcome by using information from video sequence. Finally we present the experimental results of a real video sequence