Local-Global-Learning of Naive Bayesian Classifier

Naive Bayes (NB) models are among the simplest probabilistic classifiers. However, they often perform surprisingly well in practice, even though they are based on the strong assumption that all attributes are conditionally independent given the class variable. The vast majority of research on NB models assume that the conditional probability tables in the model are either learned by maximum likelihood or Bayesian methods, even though it is well documented that learning NB models in this way may harm the expressiveness of the models. In this paper, we focus on an alternative technique for learning the conditional probability tables from data. Instead of frequency counting (which leads to maximum likelihood parameters), we propose a learning method that we call "local-global-learning". We learn the (local) conditional probability tables under the guidance of the (global) NB model learnt thus far. The conditional probabilities learned by local-global-learning are therefore geared towards maximizing the classification accuracy of the models instead of maximizing the likelihood of the training data. We show through extensive experiments that local global learning can significantly improve the classification accuracy of NB models when compared to traditional maximum likelihood learning.