The effects of algorithmic diversity on anomaly detector performance

Common practice in anomaly-based intrusion detection assumes that one size fits all: a single anomaly detector should detect all anomalies. Compensation for any performance shortcoming is sometimes effected by resorting to correlation techniques, which could be seen as making use of detector diversity. Such diversity is intuitively based on the assumption that detector coverage is different - perhaps widely different - for different detectors, each covering some disparate portion of the anomaly space. Diversity, then, enhances detection coverage by combining the coverages of individual detectors across multiple sub-regions of the anomaly space, resulting in an overall detection coverage that is superior to the coverage of any one detector. No studies have been done, however, in which measured effects of diversity amongst anomaly detectors have been obtained. This paper explores the effects of using diverse anomaly-detection algorithms in intrusion detection. Experimental results indicate that while performance/coverage improvements can in fact be effected by combining diverse detection algorithms, the gains are not the result of combining large, non-overlapping regions of the anomaly space. Rather, the gains are seen at the edges of the space, and are heavily dependent on the parameter values of the detectors, as well as on anomaly characteristics. Based on this study, defenders can be provided with knowledge of how combinations of diverse, sequence-based detectors behave to effect detection performance superior to that of a single detector.