Exploiting pattern relationship for intrusion detection

The problem of identifying patterns from system call trails of UNIX processes to better model application behavior has been investigated intensively. Most existing approaches focus on capturing the relationship between individual system calls (or system audit events). We add one additional dimension to the problem domain by also taking into consideration the overlap relationship between patterns. We first present a pattern extraction algorithm to generate maximal patterns from system call trails. Overlap relationship between patterns is subsequently investigated and stored Finally, both maximal patterns and their relationships are exploited to detect deviations from normal application behavior. We test this idea using the popular sendmail data set and the login data set obtained from the University of New Mexico. Experimental results indicate that our scheme achieves a much higher detection rate than systems that only consider intra-pattern relationship while maintaining a very low false alarm rate with similar space and time efficiency.