An Efficient Clustering Algorithm for Microblogging Hot Topic Detection

Microblog has become exceeding popular, with hundreds of millions of tweets being posted every minute on variety of topics. Most hot event will be retweeted thousands of times in short time, which will help us to trace hot event. This paper focuses on tracing those events by mining the text stream in microblog. Although event detection has long been a research topic, the characteristics of microblog bring new challenge. Tweets reporting such events are usually overwhelmed by high flood of meaningless tweets, algorithm needs to be scalable given the sheer amount of tweets. Firstly, we use Bayes classification to filter the meaningless tweets, then detect hot event from the tweets by a mean calculation based incomplete clustering. The experiments show that algorithm can detect hot events real-time from big amount tweets and remain good accuracy.