Measuring the Validity of Document Relations Discovered from Frequent Itemset Mining

The extension approach of frequent itemset mining can be applied to discover the relations among documents. Several schemes, i.e., n-gram, stemming, stopword removal and term weighting, can be applied to form different document representations for mining. It is necessary to formulate a benchmark for comparing the quality of discovered relations extracted from various document representations. This work proposes a series of evaluation criteria, called order accumulative citation matrix, which is formulated from the citation information in the publications. A new measure, called validity, is presented to reflect the validity (or quality) of discovered relations based on the proposed evaluation criteria. Regarding to the dataset, the expected validity is determined as a baseline for each set of discovered relations. With more than 10,000 documents, the experimental results show that the document relations using bigram as term definition are more valid than those using unigram with a gap of 13% to 35%. Although the term frequency weighting can improve the validity of discovered document relations when applying unigram as term definition, the binary weighting performs better in the case of bigram. Comparing to the baseline, the results show that the discovered document relations are significantly more valid than the expectation with the factor of 10 to 1,000