Sequential Pattern Mining on Highly Similar and Dense Dataset

In recent years, there are a great deal of efforts on sequential pattern mining, but some challenges have not been resolved, such as large search spaces and the ineffectiveness in handling highly similar, dense and long sequences. In this paper, we mainly focus on how to design some effective search space pruning methods to accelerate the mining process. We present a novel structure, Prefix-Frequent-Items Graph (PFI-Graph), which presents the prefix frequent items of other items in sequential patterns. An efficient algorithm, Prefix-Frequent-Items PrefixSpan (PFI-PrefixSpan) based on PFI-Graph is proposed in this paper. It avoids redundant data scanning, and thus can effectively speed up the discovery process of new patterns. Extensive experimental results on some real sequence datasets show that the proposed novel structure is substantially more efficient than PrefixSpan with pseudo-projection, especially for dense and highly similar sequence databases.