Framelet Kernels With Applications to Support Vector Regression and Regularization Networks

Support vector regression and regularization networks are kernel-based techniques for solving the regression problem of recovering the unknown function from sample data. The choice of the kernel function, which determines the mapping between the input space and the feature space, is of crucial importance to such learning machines. Estimating the irregular function with a multiscale structure that comprises both the steep variations and the smooth variations is a hard problem. The result achieved by the traditional Gaussian kernel is often unsatisfactory, because it cannot simultaneously avoid underfltting and overfltting. In this paper, we present a new class of kernel functions derived from the framelet system. A framelet is a tight wavelet frame constructed via multiresolution analysis and has the merit of both wavelets and frames. The construction and approximation properties of framelets have been well studied. Our goal is to combine the power of framelet representation with the merit of kernel methods on learning from sparse data. The proposed framelet kernel has the ability to approximate functions with a multiscale structure and can reduce the influence of noise in data. Experiments on both simulated and real data illustrate the usefulness of the new kernels.