Single image multimaterial estimation

Estimating the reflectance and illumination from a single image becomes particularly challenging when the object surface consists of multiple materials. The key difficulty lies in recovering the reflectance from sparse angular samples while correctly assigning them to different materials. We tackle this problem by extracting and fully leveraging reflectance priors. The idea is to strongly constrain the possible solutions so that the recovered reflectance conform with those of real-world materials. We achieve this by modeling the parameter space of a directional statistics BRDF model and by extracting an analytical distribution of the subspace that real-world materials span. This is used, with other priors, in a layered MRF-based formulation that models material regions and their spatially varying reflectance with continuous latent layers. The material regions and their reflectance, and the direction and strength of a single point source are jointly estimated. We demonstrate the effectiveness of the method on real and synthetic images.