Sensor-assisted image deblurring of consumer photos on smartphones

Hand-held mobile phone photography usually suffers from motion blur. Without information about the camera motion, it is difficult for traditional methods to remove such blur. We study sensor-assisted single image deblurring on modern smartphones. Information about the camera motion is obtained from the smartphones´ built-in sensors such as ac-celerometers, gyroscopes, and magnetometers. Addressing the characteristics of mobile photo capture, we propose a camera model to estimate the point spread function (PSF) based on the 3D camera orientation computed from the fused sensor data. A convenient framework is developed for automatically calibrating the camera and sensors using a series of consecutively captured photos, without requiring any extra device. In addition, to accommodate users´ sensitivity in artifacts over human faces, a face-adaptive deblurring framework is proposed. Features measuring image quality are computed over the automatically detected face regions, based on which the appropriate deblurring algorithm is selected. We evaluate our approach over 500 consumer photos featuring various levels of blur and illumination, where our algorithm shows clear advantages compared with state-of-the-art traditional blind and non-blind deconvolution methods in both de-blurring quality and speed.