Learning Bayesian networks. I. A theory based on MAP-MDL criteria

Bayesian networks provide a powerful architecture for information fusion of multiple disparate variables. A theory of learning discrete Bayesian networks from data is presented in this paper. The theory is based on a joint criterion of maximizing the joint probability or interchangeably minimizing the joint description length of the data and the Bayesian network model including the network structure and the probability distribution parameters. The computable formalisms for the data likelihood given a structure, the description length of a structure, and the estimation of the parameters given a structure are derived. EM algorithms are constructed for handling incomplete and soft data. The theory leads to a computational algorithm described in a companion paper.