Using Machine Learning for Behavior-Based Access Control: Scalable Anomaly Detection on TCP Connections and HTTP Requests

Today´s business processes are more connected than ever before, driven by the ability to share the right information with the right partners at the right time. While this interconnectedness and situational awareness is crucial to success, it also opens the possibility for misuse of the same capabilities by sophisticated adversaries to spread attacks and exfiltrate or corrupt critical sensitive information. We have been investigating means to analyze behaviors of actors and assess trustworthiness of information to support real-time cyber security decision making through a concept called Behavior-Based Access Control (BBAC). The work described in this paper focuses on the statistical machine learning techniques used in BBAC to make predictions about the intent of actors establishing TCP connections and issuing HTTP requests. We discuss pragmatic challenges and solutions we encountered in implementing and evaluating BBAC, discussing (a) the general concepts underlying BBAC, (b) challenges we have encountered in identifying suitable datasets, (c) mitigation strategies to cope with shortcomings in available data, (d) the combination of clustering and support vector machines for performing classification at scale, and (e) results from a number of scientific experiments. We also include expert commentary from Air Force stakeholders and describe current plans for transitioning BBAC capabilities into the Department of Defense together with lessons learned for the machine learning community.