Analysis of Abnormity Return in Subprogram from Malicious Executables

In recent years, the increase of malicious executables has presented a serious threat to enterprises, organizations, and individuals. In order to avoid being analyzed statically, malicious codes resort to various obfuscation techniques to hide their malicious behaviors. The technique based on the abnormity return of subprogram is one of the techniques. The disassemblers, such as IDAPro and OBJDump, couldn´t deal with malware which uses this technique. This paper describes the principles adopted by a malware to implement the exception return in the subprogram, and presents an extended disassembly algorithm for handling this kind of malware. The capability of the disassembly algorithm is analyzed and tested. The result of the test proves that the algorithm is effective.