A System for Ranking Organizations Using Social Scale Analysis

In this paper we utilize feature extraction and model fitting techniques to process the rhetoric found in the web sites of 23 Indonesian religious organizations - comprising a total of 37,000 articles dating from 2005 to 2011 - to profile their ideology and activity patterns along a hypothesized radical/counter-radical scale. We rank these organizations by assigning them to probable positions on the scale. We show that the developed Rasch model fits the data using Andersen´s LR-test. We create a gold standard of the ranking of these organizations through an expertise elicitation tool. We compute expert-to-expert agreements, and we present experimental results comparing the performance of three different baseline methods to show that the Rasch model not only outperforms our baseline methods, but it is also the only system that performs at expert-level accuracy.