Real-Time Hybrid Intrusion Detection System Using Apache Storm

Networks are prone to intrusions and detecting intruders on the Internet, is a major problem. Many Intrusion Detection Systems have been proposed to detect these intrusions. However, as the Internet grows day by day, there is a huge amount of data (big data) that needs to be processed to detect intruders. For this reason, intrusion detection has to be done in real-time before intruders can inflict damage, and previous detection systems do not satisfy this need for big data. Using Apache Storm, a Real time Hybrid Intrusion Detection System has been developed in our work. Apache Storm serves as a distributed, fault tolerant, real time big data stream processor. The hybrid detection system consists of two neural networks. The CC4 instantaneous neural network acts as an anomaly-based detection for unknown attacks and the Multi Layer Perceptron neural network acts as a misuse-based detection for known attacks. Based on the outputs from these two neural networks, the incoming data will be classified as "attack" or "normal." We found the average accuracy of hybrid detection system is 89% with a 4.32% false positive rate. This model is appropriate for real time detection since Apache Storm acts as a real time streaming processor, which can also handle big data.