Leveraging State Information for Automated Attack Discovery in Transport Protocol Implementations

We present a new method for finding attacks in unmodified transport protocol implementations using the specification of the protocol state machine to reduce the search space of possible attacks. Such reduction is obtained by appling malicious actions to all packets of the same type observed in the same state instead of applying them to individual packets. Our method requires knowledge of the packet formats and protocol state machine. We demonstrate our approach by developing SNAKE, a tool that automatically finds performance and resource exhaustion attacks on unmodified transport protocol implementations. SNAKE utilizes virtualization to run unmodified implementations in their intended environments and network emulation to create the network topology. SNAKE was able to find 9 attacks on 2 transport protocols, 5 of which we believe to be unknown in the literature.