A Testing Model for Dynamic Malware Analysis Systems

A Dynamic Malware Analysis System (D-MAS), often called a sandbox, is a controlled environment in which malicious software (malware) is executed in order to identify the actions it is performing (e.g., creating processes, sending emails) when infecting computer systems. One of the most important features of security devices such as IDSs, AVSs and D-MASs, is how accurately they identify and document threats. By nature, these security devices are difficult to test since they are test systems themselves. The attackers are the testers trying to find test cases that cannot be identified by these systems. Consequently, thorough testing models are required by developers to assess the accuracy of D-MASs, an area in which very little theoretical and empirical work exists. In this paper, we lay out the basis of D-MASs accuracy assessment and we present an evaluation of eight of these systems. We propose test coverage criteria, oracle types and specifications to assess the accuracy of D-MASs. Results show that our approach is efficient at identifying accuracy problems in several D-MASs.