Evaluating Machine Learning and Unsupervised Semantic Orientation approaches for sentiment analysis of textual reviews

This paper presents our experimental work on evaluation of Machine Learning based classification approaches (Naïve Bayes and SVM) with the Unsupervised Semantic Orientation based SO-PMI-IR algorithm for sentiment analysis of movie review texts. We have used both pre-existing data sets and our own dataset collection comprising of a large number of user reviews for Hindi movies. The Naïve Bayes and SVM approaches were implemented in multiple folds. The results, in addition to presenting a detailed comparative view of these techniques, demonstrate that with suitable selection of features the Naive Bayes algorithm performs reasonably well and at times matches the popularly believed superior performance level of SVM, at least for sentiment analysis task. The SO-PMI-IR algorithm produces substantially accurate sentiment classification without the requirement of any prior training. The accuracy of SO-PMI-IR however depends on POS tags used as features and thresholding/ aggregation schemes used.