Optimizing Network Anomaly Detection Scheme Using Instance Selection Mechanism

Network anomaly detection is a classically difficult research topic in intrusion detection. However, existing research has been solely focused on the detection algorithm. An important issue that has not been well studied so far is the selection of normal training data for network anomaly detection algorithm, which is highly related to the detection performance and computational complexity. Based on our previous proposed TCM-KNN (Transductive Confidence Machines for K-Nearest Neighbors) anomaly detection method, which can detect anomalies with high detection rate and low false positive rate, we develop an instance selection mechanism for TCM-KNN based on EFCM (Extended Fuzzy C-Means) clustering algorithm in this paper, aiming at limiting the size of training dataset, thus reducing the computational cost of TCM-KNN and boosting its detection performance. We report the experimental results over real network traffic. The results demonstrate the instance selection method presented in this paper is effective for TCM-KNN and thus optimizing it as an effectively lightweight network anomaly detection scheme.