Cross-Modality Sentiment Analysis for Social Multimedia

Sentiment analysis is important for understanding the social media contents and user opinions. Along with the development of social media applications, an increasing number of people combine texts and images to express their opinions. However, text based sentiment analysis methods cannot process other medias except texts. Therefore, visual sentiment analysis is born at the right moment. In this article, we review two multimodal-based visual sentiment analysis models proposed in our group. Both model exploit the multimodal content from correlation and hyper graph view respectively. In the Multimodal Correlation Model (MCM), we observe the correlation among different modalities and model then through a probabilistic graphical model. In the Hyper graph Learning Model (HLM), we use hyper graph to model the independence of each modality. We further discuss the underneath challenges and foresee potential opportunities of this direction.