One data preprocessing method in high-speed network Intrusion Detection

With the development and popularization of highspeed network technology, the Intrusion Detection System need to process more and more data. In most cases, only a small part of the data which need to be processed is the abnormal data. The abnormal data will be overwhelmed by the normal data. A large number of normal data will occupy most of resources of the IDS and lead to a lot of false alarms. These all will bring troubles for Intrusion Response System and administrators. In this paper, we present a method based on semi-supervised learning to process the massive data in the high-speed network. We add some representative and labeled data to the large unlabeled dataset, and let them cluster. We regard the data which is most similar to the labeled data as normal data. It is suggested that the method can reduce the redundant data and false alarms efficiently, and also can improve the computation time.