Investigating accuracies of rule evaluation models on randomized labeling and human evaluation

In datamining post-processing, rule selection using objective rule evaluation indices is one of a useful method to find out valuable knowledge from mined patterns. However, the relationship between an index value and expertspsila criteria has never been clarified. In order to determine the relationship, we have developed a method to obtain learning models from a dataset consisting of objective rule evaluation indices and evaluation labels for rules. In this study, we have compared the accuracies of classification learning algorithms for datasets with randomized class distributions. Then, the results show that accuracies of classification learning algorithms with/without criteria of human experts are different on a balanced randomized class distribution.