A binary analysis method for protocol deviation discovery from implementations

Generally there are several versions of implementations for a network protocol specification. However, these protocol implementations inevitably contain deviations, which are usually introduced by errors in engineering development process or different interpretations of software developers for the same protocol specification. Thus, discovery of protocol deviations in implementation binaries is significant for several applications, such as exploiting vulnerability and generating fingerprint from a series of protocol implementations. In this paper, we propose an approach to discover deviations based on dynamic binary analysis technique and semantic-based active testing technique. Comparing with previous similar works, the proposed approach has the following advantages: (1) It works on the binaries of protocol implementations under test, thus do not have to require source code for each implementation; (2) By observing how protocol implementation parses encrypted messages, it is able to discover deviations in cryptographic protocol implementations. We have developed a prototype system to evaluate the proposed approach, and use this system to discover deviations in several implementations of two kinds of protocols: text-based HTTP protocol and cryptography-based FTPS protocol. The results show that the prototype system can successfully discover deviations between their different implementations, and verifies the feasibility of the proposed approach.