AntiBot: Clustering Common Semantic Patterns for Bot Detection

Among malicious software (malware), autonomous malicious programs, called bots, are a serious problem in the Internet. The bot writers have developed a variety of techniques to evade simple signature-based detection. Concise representations of malware behavior, or semantic patterns, are much harder to evade or obfuscate. However, generating a semantic pattern for every program instance is time-consuming, and comparing with a large number of patterns creates a challenge for timely identification of bots. This paper proposes an automated approach to generate semantic patterns for bot detection. Unlike previous approaches, it is intended to find one pattern that accurately represents the important behavior of an entire class of bots, rather than of individual instances. Doing so has advantages for fast malware identification, and for distinguishing new classes of attacks from previously-seen attacks. The work uses static analysis to characterize bot behaviors, and proposes to use hierarchical clustering of the resulting semantic patterns from a set of bot programs. The goal is to identify critical, common semantic behavior that represents the functions of an entire class of the malware. This method has been prototyped and evaluated on real-world malicious bot software. Depending on parameter choices, our approach can achieve more than 95% detection rates and less than 5% false positive rates on a large set of bot programs and non-bot executables.