Realization and evaluation of a decaying co-occurrence model based on parallel computing

This paper proposes and realizes a decaying co-occurrence model, which can generate the thesaurus of similarity between words automatically. The model can get a more precise thesaurus because it considers the distance between words, which is unlike the common co-occurrence model. But its huge computation makes it impracticable in a very large corpus. In order to overcome this limitation, a parallel computing system for calculating the decaying co-occurrence model has been developed, and has been running successfully on the MPI cluster environment. High speedup and efficiency are obtained. This is a substantive step for the construction of basics resource of Chinese information processing.