A real-time multi-cue framework for determining optical flow confidence

In recent years, many dense optical flow algorithms have been presented. Besides the actual result, the confidence of the result is of uttermost importance for safety-critical applications using flow. We focus on the determination of the confidence of optical flow. The main contribution of this paper is the introduction of a real-time multi-cue framework for determining optical flow confidence. We use a Mixture-of-Gaussians model (GMM) and train a classifier to learn the contribution of each individual cue. In addition we introduce two new confidence measures based on spatial and temporal optical flow variances. The actual confidence image can be computed in less than 13ms on a graphics processing unit (GPU). Results on the Urban scene from the Middlebury database as well as results on rendered traffic scenes with known ground truth show a significant improvement of the confidence compared to single cues using the percentage- of-retained-pixel or sparsification measure. Some results on real-world scenes trained with rendered data show a good visual agreement and confirm the generality of the approach. We present results obtained with the well-known TV-L¹ flow method and with an alternative flow method to prove the applicability to different types of optical flow.