Support vector machines learning noisy polynomial rules

Using statistical physics, we study support vector machines (SVMs) learning noisy target rules in cases when the optimal predictor is a polynomial of the inputs. If the kernel of the SVM has sufficiently high order or is transcendental, the scale of the learning curve and the asymptote is determined by the target rule and does not depend on the kernel. On this scale we find convergence to optimal generalization but no convergence of the training error to the generalization error.