Early detection of high entropy traffic

High entropy (HE) traffic may result from encrypted traffic such as C&C botnet communication. Such traffic also tends to be opaque to an IDS. However, line speed entropy calculation is expensive, especially for long flows. In this paper we introduce methodology to classify flows as HE or low entropy (LE) by considering only the first M packets of the flow. We use our HE classifiers in two ways: (a) to improve the effectiveness of BotHunter, a bot detection tool when presented with encrypted bot traffic, and (b) as a filter to reduce the load on an IDS. We implement our filter as a Snort preprocessor. Our results show that integration with BotHunter improves detection drastically. When used as a filter, our classifiers reduce the amount of traffic delivered to IDS by more than 50%, while maintaining more than 99.9% of the original alerts. Other work needs to inspect at least 13 times more packets or it misses about 70 times of the alerts.