A constrained nonlinear energy minimization framework for the regularization of the stereo correspondence problem

In this paper, we propose a novel approach to stereo correspondence based on the optimization of a continuous disparity surface defined parametrically using radial basis functions. Principal advantages over other methods include the use of constrained nonlinear programming to perform regularization as a hierarchical multiobjective optimization which differs from the standard weighted sum approach, so that regularization becomes more consistent with the notion of Pareto optimality. Furthermore, the optimization algorithm is capable of handling arbitrary constraints on the sought parameters, so that a variety of types of a priori scene information can be incorporated explicitly to the problem definition. To exemplify this we derive a new continuous unary formulation of the disparity gradient limit constraint and propose other types of potential constraints for a priori knowledge. Furthermore, the optimization employs a smoothness oriented regularization operator to preserve surface discontinuities, a flexible block decomposition approach of the disparity surface to allow parallelization and a correlation-based fitting with heuristics to initialize the parameters and avoid local optima effectively. Experiments with standard stereo imagery show that the method handles adequately the imposed constraints and produces surfaces with accurate level of elevation detail.