Title :
Naïve Bayesian filters for log file analysis: Despam your logs
Author :
Havens, Russel W. ; Lunt, Bary ; Teng, Chia-Chi
Author_Institution :
Sch. of Technol., Brigham Young Univ., Provo, UT, USA
Abstract :
System log files are critical for troubleshooting complex modern computer systems. Systems can easily produce more log file entries than a human can realistically use. However, there are a number of good filtering and clustering technologies that are used in various areas of data mining. This research focuses on using very easily accessible Bayesian spam filters for categorizing log entries. Results of this research have confirmed that these filters can be effectively used to discover log entries related to known issues, and to effectively disprove outage relationships. Both of these techniques can be easily instrumented in a log analysis framework and provide administrators with much needed filtering for similar logs and thus, similar outages.
Keywords :
Bayes methods; data mining; information filtering; information filters; pattern clustering; system monitoring; unsolicited e-mail; Bayesian spam filters; Naive Bayesian filters; clustering technologies; computer systems; data mining; log entries categorization; log file analysis; outage relationships; system log files; troubleshooting; Bayesian methods; Information filters; Noise; Servers; Springs; Bayesian content filter; Bogofilter; Spam Assassin; Spam Bayes; log file analysis; spam filter; word chaining;
Conference_Titel :
Network Operations and Management Symposium (NOMS), 2012 IEEE
Conference_Location :
Maui, HI
Print_ISBN :
978-1-4673-0267-8
Electronic_ISBN :
1542-1201
DOI :
10.1109/NOMS.2012.6211972