Call-Flow Aware API Fuzz Testing for Security of Windows Systems

API fuzz testing is to insert unexpected data into parameters of functions and to monitor exceptions or errors of a software system in order to test security of it. API fuzz testing without considering the dependency between functions generates many errors, because required functions aren´t called before the target function is called. Therefore, unexpected data cannot reach various codes in the target function. We define the dependency as relation of functions that must be called before the target function is called. In order to solve the problem of the dependency during performing API fuzz testing, we propose a novel Methodology that analyzes the dependency between functions automatically and that performs API fuzz testing with considering the dependency, and implement a practical tool for our methodology. We name the methodology the Call-Flow Aware API Fuzz Testing (CFAFT). Call-Flow is the order that functions with the dependency are called. By considering Call-Flow of functions, CFAFT can perform API fuzz testing without errors related to the dependency and insert invalid data into various codes in functions. We experimented on DLL files in the system folder of Windows XP SP2. Experimental result showed that CFAFT removed errors related to the dependency between functions.