A semantic set theory for word semantic similarity assessment

A core issue for the vector space model based semantic similarity assessing algorithms is how to weight dimensional values for a headword. In this paper, a semantic set is supposed to be existed. Weight functions are in fact converting functions used to project the sematic set to practical vector spaces. For the converting property, a proper weight function should be non-linear and unrelated dimensions insensitive. Following this idea, a mutual information based weight function is proposed. With this function, most of unrelated dimensions for a headword could be filtered out and the values of remaining dimensions could be well weighted to minimize the effects caused by non-semantic factors such as grammatical relations and pragmatic habits. Experiments show that the new weight function performs better compared with the other metrics. These results also state the reasonability of the semantic set theory.