Anomalous Payload Detection System Using Analysis of Frequent Sequential Pattern

We present a new framework of anomalous payload detection system. First of all, frequent sequential patterns (FSPs) are mined from raw traffic payloads. Setting different supports, we have several level of description of normal payload. We extract each FSP feature using n-gram technique. Thus we can have a deeper insight of data flow. By using advanced clustering method to fulfill the feature reduction, we obtain a compact representative dataset which can be directly used to intelligent system. One-class SVM classifier is used to construct each detector and ensemble method is used to further improve the performance of system. Experimental results show that our anomalous payload detection system can effectively detect the mimicry attack and other stealthy exploits.