Mutual Information Between Random Processes from High Dimensional Data

A number of signal estimation problems are arising where a relatively low dimensional state is to be estimated from a high dimensional observation sequence. In previous work we have shown this leads to considerable simplification in the structure of optimal state estimators even in non-linear problems. In these and other state estimation problems there is a growing interest in the computation of mutual information between unobserved state and observed sequence. Here we show that the mutual information computation can be likewise considerably simplified