Diffusion Information Theoretic Learning for Distributed Estimation Over Network

Distributed estimation over networks has received a lot of attention due to its broad applicability. In diffusion type of distributed estimation, the parameters of interest can be well estimated from noisy measurements through diffusion cooperation between nodes. Meanwhile, the consumption of communication resources is low, since each node exchanges information only with its neighbors. In previous studies, most of the cost functions used in diffusion distributed estimation are based on mean square error (MSE) criterion, which is optimal only when the measurement noise is Gaussian. However, this condition does not always hold in real-world environments. In non-Gaussian cases, the information theoretic learning (ITL) provides a more general framework and has a better performance than the MSE-based method. In this work, we incorporate information theoretic measure into the cost function of diffusion distributed estimation. Moreover, an information theoretic measure based adaptive diffusion strategy is proposed to further promote estimation performance. Simulation results show that the diffusion ITL-based distributed estimation method can achieve superior performance comparing to the standard diffusion least mean square (LMS) algorithm when the noise is modeled to be non-Gaussian.