Glass object localization by joint inference of boundary and depth

We address the problem of localizing glass objects with a multimodal RGB-D camera. Our method integrates the intensity and depth information from a single view point, and builds a Markov Random Field that predicts glass boundary and region jointly. Based on the localization, we also reconstruct the depth of the scene and fill in the missing depth values. The efficacy of our algorithm is validated on a new RGB-D Glass dataset of 43 distinct glass objects.