Automatic word classification using simulated annealing

A bigram class model which gives the probability of a word class given its predecessor class has been developed. Simulated annealing is used to classify automatically the words of large text corpora. A first validation of the use of simulated annealing in language modeling is presented. Results are presented using a French corpus of 40000 words and a German corpus of 100000 words. It is demonstrated that simulated annealing makes it possible to classify words without any syntactic or semantic knowledge. The best results are obtained with all words gathered into a unique class at the beginning of the optimization. Simulated annealing is easy to implement and CPU time cost is not prohibitive: seven hours on a 486-33 MHz PC to classify 14000 words into 120 classes using a 75000 word training set, without any code optimization.<>