The past is important: a method for determining memory structure in NARX neural networks

Recurrent networks have become popular models for system identification and time series prediction. NARX (nonlinear autoregressive models with exogenous inputs) network models are a popular subclass of recurrent networks and have been used in many applications. Though embedded memory can be found in all recurrent network models, it is particularly prominent in NARX models. We show that the use of intelligent memory order selection through pruning and good initial heuristics significantly improves the generalization and predictive performance of these nonlinear systems on problems as diverse as grammatical inference and time series prediction