Automatic visual analysis of real-world events covered by social media using convolutional neural networks

This paper investigates how well real-world events can be characterized by visual features detected in related images posted on social media, using state-of-the-art computer vision methods for object detection and classification. Over 48k images from four different events have been processed to detect objects of different types using convolutional neural networks (CNNs) and cascaded classifiers. Based on these object detections we train different classifiers to rank object types supporting the respective event and to discriminate images of an event from other images. Possible applications include: (1) finding images of a certain event in a semi-automatic way, and (2) classifying the type of an event.