Malware Obfuscation Detection via Maximal Patterns

Malware obfuscation is defined as a program transformation. It is always used in malware to evade detection from anti-malware software. In this paper, we propose a method to detect malware obfuscation using maximal patterns. Maximal pattern is a subsequence in malware´s runtime system call sequence, which frequently appears in program execution, and can be used to describe the program specific behavior. The maximal pattern sequence is extracted from the malware´s runtime system calls, and the similarity between two pattern sequences will be measured by evolutionary similarity. Based on the real-world malwares test data, the experiment results have shown that our method can efficiently detect malware obfuscation.