A New Coupled Metric Learning for Real-time Anomalies Detection with High-Frequency Field Programmable Gate Arrays

Billions of internet end-users and device to device connections contribute to the significant data growth in recent years, large scale, unstructured, heterogeneous data and the corresponding complexity present challenges to the conventional real-time online fraud detection system security. With the advent of big data era, it is expected the data analytic techniques to be much faster and more efficient than ever before. Moreover, one of the challenges with many modern algorithms is that they run too slowly in software to have any practical value. This paper proposes a Field Programmable Gate Array (FPGA) -based intrusion detection system (IDS), driven by a new coupled metric learning to discover the inter- and intra-coupling relationships against the growth of data volumes and item relationship to provide a new approach for efficient anomaly detections. This work is experimented on our previously published NetFlow-based IDS dataset, which is further processed into the categorical data for coupled metric learning purpose. The overall performance of the new hardware system has been further compared with the presence of conventional Bayesian classifier and Support Vector Machines classifier. The experimental results show the very promising performance by considering the coupled metric learning scheme in the FPGA implementation. The false alarm rate is successfully reduced down to 5% while the high detection rate (=99.9%) is maintained.