Iterative design of regularizers based on data by minimizing generalization errors

In our previous study (1998) we proposed a theoretical evaluation of generalization errors. However, it suffered from from serious difficulties: 1) it assumes that true model parameters and noise variance are known a priori; and 2) it assumes that input variables are mutually independent. These assumptions prevent its application to real data. The present paper succeeds in overcoming these two difficulties. A key idea is to iteratively estimate these parameters and generalization errors from data. Introducing correlations between input variables is not intrinsically difficult, although it makes computation much more complex than the cases where input variables are mutually independent