Quick asymmetric text similarity measures

Text similarity measure is a common issue in information retrieval, text mining, Web mining, text classification/clustering and document copy detection etc. The most popular approach is word frequency based scheme, which uses a word frequency vector to represent a document. Cosine function, dot product and proportion function are regular similarity measures of vector. But they are symmetric similarity measures, which cannot find out subset copies. In this paper we present the concepts of asymmetric similarity model and heavy frequency vector (HFV). The former can detect subset copies well; the latter can save a great resources and CPU time. We develop two new asymmetric measures: HFM and HIPM. The HFM and HIPM are derived from cosine function and proportion function by combining asymmetric similarity concept with HFV. The HFV is to truncate the original full frequency vector to a short vector. We can adjust the parameter of HFV to balance the model´s performance. Several experiments illustrate aspects of asymmetric similarity and HFV models in this paper.