Spatio-temporal adaptive 3-D Kalman filter for video

This paper presents three-dimensional (spatio–temporal) Kalman filters for video as the extension of the two-dimensional (2-D) reduced update Kalman filter (RUKF) approach for images. We start out with threedimensional (3-D) RUKF, a shift-invariant recursive estimator with efficiency advantages over the 3-D Wiener filter. Then, we turn to the motion-compensated extension MC-RUKF, which gives improved performance when coupled with a motion estimator. Since motion compensation sometimes fails, causing severe fluctuations in temporal correlation, we then present multimodel MC-RUKF, to adapt to variation in temporal and spatial correlation, by detecting the local image model out of a class, and using it in MC-RUKF. Finally, we introduce a novel multiscale model detection algorithm for use in high noise environments.