PsybOt malware: A step-by-step decompilation case study

Decompilation (i.e. reverse compilation) represents one of the most toughest and challenging tasks in reverse engineering. Even more difficult task is the decompilation of malware because it typically does not follow standard application binary interface conventions, has stripped symbols, is obfuscated, and can contain polymorphic code. Moreover, in the recent years, there is a rapid expansion of various smart devices, running different types of operating systems on many types of processors, and malware targeting these platforms. These facts, combined with the boundedness of standard decompilation tools to a particular platform, imply that a considerable amount of effort is needed when decompiling malware for such a diversity of platforms. This is an experience paper reporting the decompilation of a real-world malware. We give a step-by-step case study of decompiling a MIPS worm called psyb0t by using a retargetable decompiler that is being developed within the Lissom project. First, we describe the decompiler in detail. Then, we present the case study. After that, we analyse the results obtained during the decompilation and present our personal experience. The paper is concluded by discussing future research possibilities.