Unsupervised context learning in natural language processing

By generalizing over contextual information, excellent results were obtained in connectionist language processing. Normally, these contexts are added manually to the system or deducted by using a supervised learning algorithm. A recurrent self-organizing model, capable of deriving the context from scratch, is presented. Syntactic features and structures are learned in a unsupervised way from flat sentences. By generalizing over the words as well as the sentences, simple semantics can be derived. The model forms a two-layer extension of the Kohonen feature map, provided with additional recurrent fibers which are responsible for the automatic determination of word contexts, thus resulting in an unsupervised recurrent learning algorithm. After a formal description of the model, the experimental results are presented