Assessing generalization of feedforward neural networks

Neural networks have been used to tackle what might be termed `empirical regression´ problems. Given independent samples of input/output pairs (x_i,y_i), we wish to estimate f(x)=E[Y|X=x]. The approach taken is to choose an approximating class of networks N={η(x;w)}w∈W and within that class, by an often complex procedure, choose an approximating network η(·;w*). The distance (in mean squared error) of this network from f can be separated into two terms: one for approximation or bias-choosing N large enough so that some η(·;w⁰), say, models f well-and one for estimation or variance-how well the chosen η(·;w*) performs relative to η(·;w⁰). We address the latter term