Iterative algorithms for optimal signal reconstruction and parameter identification given noisy and incomplete data

We present a new approach to the problem of estimating multiple signal and parameter unknowns given noisy and incomplete data. Using cross-entropy, we fit a separable density to the given model density, then use this separable density to estimate each unknown independently. Not only does this method include all the various MAP methods as degenerate cases, but it also directly leads to a simple iterative algorithm which can solve either the cross-entropy method or any of the MAP methods. This algorithm is particularly effective for exponential families of densities. Applications include estimation using grouped or quantized data, and a wide variety of reconstruction, smoothing, interpolation, extrapolation and modeling problems involving linear Gaussian systems.