A Noisy-Or Discriminative Restricted Boltzmann Machine for Recognizing Handwriting Style Development

Restricted Boltzmann machines (RBMs) and their variants have attracted a lot of attention recently. They have been applied widely, e.g., In handwriting recognition, document categorization and object recognition. Unfortunately, an RBM requires a large parameter space since it is a fully-connected bipartite graph, especially with high dimensional input spaces. Moreover, it is still unclear how it selects discriminative features for classification problems. This necessitates the selection of effective and discriminative features for recognition. In this paper, we propose a Noisy-Or discriminative restricted Boltzmann machine (Abbr. As NDRBM or Noisy-Or RBM), which combines RBM and Noisy-Or gate function. On the one hand, it can greatly reduce the parameter space. Furthermore, this model extends the RBM into a multiple instance learning scenario-to help select discriminative features or regions. An approximate approach is proposed to make the NDRBM practical. We apply our method on handwriting style development recognition and recognition rates are observed to be better than competitive baselines on two data sets (cursive and handprint data from Grades 2-4).