VC dimension theory for a learning system with forgetting

In a changing environment, forgetting old samples is an effective method to improve the adaptability of learning systems. However, too fast forgetting causes a decrease of generalization performance. In this paper, we analyze the generalization performance of a learning system with a forgetting parameter. For a class of binary discriminant functions, it is proved that the generalization error is given by O(√hα) (O(hα) in a certain case), where h is the Vapnik-Chervonenkis (VC) dimension of the class of functions and 1-α represents a forgetting rate. The result provides a criterion to determine the optimal forgetting rate.