Discriminative Multi-stream Discrete Hidden Markov Models

We propose a modified discrete HMM that handles multimodalities. We assume that the feature space is partitioned into subspaces generated by different sources of information. To combine these heteregoneous modalities we propose a multi-stream discrete HMM that assigns a relevance weight to each subspace. The relevance weights are set local and depend on the symbols and the states. In particular, we associate a partial probability with each symbol in each subspace. The overall observation state probability is then computed as an aggregation of the partial probabilities and their objective relevance weights based on a linear combination. The minimum classification error (MCE) objective based on the gradient probabilistic descent (GPD) optimization algorithm is reformulated to derive the update equations for the relevance weights and the partial state probabilities. The proposed approach is validated using synthetic and real data sets. The results are shown to outperform the baseline discrete HMM that treats all streams equally important.