Towards the automated detection of unknown malware on live systems

In this paper, we propose a new system monitoring framework that can serve as an enabler for automated malware detection on live systems. Our approach takes advantage of the increased availability of hardware assisted virtualization capabilities of modern CPUs, and its basic novelty consists in launching a hypervisor layer on the live system without stopping and restarting it. This hypervisor runs at a higher privilege level than the OS itself, thus, it can be used to observe the behavior of the analyzed system in a transparent manner. For this purpose, we also propose a novel system call tracing method that is designed to be configurable in terms of transparency and granularity.