Learning context-free grammar with enhanced neural network pushdown automaton

Previously (C. Sreerupa Das et al., 1992, 1993; C.L. Giles et al., 1990; G.Z. Sun et al., 1990, 1991), a model was developed of neural network pushdown automata (NNPDA). NNPDA is a hybrid system which couples the neural network finite controller with an external continuous stack memory. It learns context-free grammars from examples by minimizing the properly defined objective function. In the original version of the NNPDA, the neural network controller is built with high-order connected recurrent neural nets. However, due to the complexity of the learning surface, this model could not learn several non-trivial grammars, among which the Palindrome grammar is considered a very difficult one. The author analyzes the difficulty of original model and makes several modifications and thus enhanced the learning power of NNPDA. The major enhancement is the introduction of a linear `full order´ recurrent neural network as the stack controller