Bikers Are Like Tobacco Shops, Formal Dressers Are Like Suits: Recognizing Urban Tribes with Caffe

Recognition of social styles of people is an interesting but relatively unexplored task. Recognizing "style" appears to be a quite different problem than categorization, it is like recognizing a letter\´s font as opposed to recognizing the letter itself. Similar-looking things must be mapped to different categories. Hence a priori it would appear that features that are good for categorization should not be good for style recognition. Here we show this is not the case by starting with a convolutional deep network pre-trained on Image Net (Caffe), a categorization problem, and using the features as input to a classifier for urban tribes. Combining the results from individuals in group pictures and the group itself, with some fine-tuning of the network, we reduce the previous state of the art error by almost half, going from 46% recognition rate to 71%. To explore how the networks perform this task, we compute the mutual information between the Image Net output category activations and the urban tribe categories, and find, for example, that bikers are well categorized as whiptail lizards by Caffe, and that better recognized social groups have more highly-correlated Image Net categories. This gives us insight into the features useful for categorizing urban tribes.