Temporal Distributed Learning with Heterogeneous Data Using Gaussian Mixtures

Learning a model for data in a distributed source system has often been performed by collecting all data at a central location and performing the learning process on the global data set at the central location. Although a common global feature space is normally assumed, each local source may only sample a subset of features, producing a heterogeneous data combination at the central location. Additionally, various constraints such as communication limitations and data privacy concerns require that limited information from each local source be sent to the central processor. The challenge is then to learn the most accurate global data model given this constrained information. In online systems, the data may be non-stationary, requiring explicitly dynamic modeling. We have proposed an online dynamic method to learn the probability distribution of a global data set as a Gaussian mixture model given synchronous updates of distribution parameters from local data sources of possibly non-overlapping features.