Large-scale socio-demographic pattern discovery on microblog metadata

Microblogging services, such as Twitter, generate huge volumes of data reflecting the current zeitgeist. As such they are of enormous potential value to studies ranging from data mining to social anthropology. To realize the potential, this study investigates improvements of algorithms specifically tailored for the discovery of latent socio-demographic patterns in Twitter metadata. These newly improved hybrid algorithms improve on existing ones in terms of speed and scalability (from thousands of records to millions). Testing on a real-world Twitter data set (~7.4 million messages) reveals emergent patterns in global day-to-day Twitter activity. The results demonstrate novel insight when applied to real-world Twitter data, practical large-scale applications of the methods, and suggest potential areas of future research.