Probabilistic Exploitation of the Lucas and Kanade Smoothness Constraint

The basic idea of Lucas and Kanade is to constrain the local motion measurement by assuming a constant velocity within a spatial neighborhood. We reformulate this spatial constraint in a probabilistic way assuming Gaussian distributed uncertainty in spatial identification of velocity measurements and extend this idea to scale and time dimensions. Thus, we are able to combine uncertain velocity measurements observed at different image scales and positions over time. We arrive at a new recurrent optical flow filter formulated in a dynamic Bayesian network applying suitable factorisation assumptions and approximate inference techniques. The introduction of spatial uncertainty allows for a dynamic and spatially adaptive tuning of the constraining neighborhood. Here, we realize this tuning dependent on the local structure tensor of the intensity patterns of the image sequence. We demonstrate that a probabilistic combination of spatiotemporal integration and modulation of a purely local integration area improves the Lucas and Kanade estimation.