A New Approach to Knowledge-Based Design of Recurrent Neural Networks

A major drawback of artificial neural networks (ANNs) is their black-box character. This is especially true for recurrent neural networks (RNNs) because of their intricate feedback connections. In particular, given a problem and some initial information concerning its solution, it is not at all obvious how to design an RNN that is suitable for solving this problem. In this paper, we consider a fuzzy rule base with a special structure, referred to as the fuzzy all-permutations rule base (FARB). Inferring the FARB yields an input-output (IO) mapping that is mathematically equivalent to that of an RNN. We use this equivalence to develop two new knowledge-based design methods for RNNs. The first method, referred to as the direct approach, is based on stating the desired functioning of the RNN in terms of several sets of symbolic rules, each one corresponding to a subnetwork. Each set is then transformed into a suitable FARB. The second method is based on first using the direct approach to design a library of simple modules, such as counters or comparators, and realize them using RNNs. Once designed, the correctness of each RNN can be verified. Then, the initial design problem is solved by using these basic modules as building blocks. This yields a modular and systematic approach for knowledge-based design of RNNs. We demonstrate the efficiency of these approaches by designing RNNs that recognize both regular and nonregular formal languages.