vMON: An Efficient Out-of-VM Process Monitor for Virtual Machines

Cloud computing service has been evolved in providing a whole virtual data center from selling scattered virtual machines (VMs). Process Monitoring of a VM is a fundamental feature to guarantee the security of the virtual data center because of the rapid growth of the malware. Existing approaches are mainly based on virtual machine introspection (VMI) technique to isolate the monitor out-of-vm and designed to inspect the VM internal processes. However, few of them consider the real time control of process execution in the VMs, such as process termination or files operation conducted by the process. Early VMI-based solutions relied on some specific OS kernel data structures, so they need to know the OS information in advance instead of identifying the OS version at runtime for operating system compatible. In this paper, we propose a novel out-of-the-box process monitor named vMON, which can not only identify different guest OS versions and reconstruct rich semantic information for the target VM processes at runtime, but also control the behaviors of processes with fine granularity. In addition, vMON provides uniform programming interfaces to support the development of application-level security tools. A prototype of vMON has been implemented in kernel-based virtual machine (KVM) hyper visor, and its effectiveness and performance have also been evaluated through several experiments. The results show that vMON can successfully identify, analyze and control the behaviors of the processes in Guest OS with acceptable performance overhead. vMon incurs 0.74%~10.20% I/O overhead and 0.003s average interface return time.