A new incremental method for function approximation using feed-forward neural networks

A sequential method for approximating vectors in Hilbert spaces, called sequential approximation with optimal coefficients and interacting frequencies (SAOCIF), is presented. SAOCIF combines two key ideas. The first one is the optimization of the coefficients. The second one is the flexibility to choose the frequencies. The approximations defined by SAOCIF maintain orthogonal-like properties. The theoretical results obtained prove that, under reasonable conditions, the residue of the approximation obtained with SAOCIF (in the limit) is the best one that can be obtained with any subset of the given set of vectors. In the particular case of L², it can be applied to approximations by algebraic polynomials, Fourier series, wavelets and feed-forward neural networks, among others. Also, a particular algorithm with feed-forward neural networks is presented. The method combines the locality of sequential approximations, where only one frequency is found at every step, with the globality of non-sequential ones, where every frequency interacts with the others. Experimental results show a very satisfactory performance