A change detector for mining frequent patterns over evolving data streams

Mining data streams for frequent patterns is important in many applications. Unlike traditional static databases, the underlying process that generates the data streams evolves over time. Past data may become outdated and of little use when compared to the most recent one. When a significant change occurs, much harm is done to the mining result if it is not properly handled. In this paper, an online algorithm for change detection in frequent pattern mining is proposed. Although there have been several studies mainly for adapting to changes, we contend that this is not enough. The ability to detect and characterize change is essential in many applications. A novel test strategy is designed to gather the ldquoevidencerdquo sufficient to conclude on whether the current sample differ significantly from a reference sample. Different statistical tests are evaluated and our study shows that the chi-square test is the most suitable for enumerated or count data.