Sequential relevance vector machine learning from time series

This paper presents an approach to sequential training of the relevance vector machine suitable for Bayesian learning from time series. The key idea is to perform simultaneous incremental optimization of both the weight parameters and their prior hyperparameters using data arriving successively one at a time. Algorithms for efficient sequential regularized dynamic learning rate training of the weights and gradient-descent training of their corresponding individual priors are derived. It is shown that this fast sequential RVM can outperform similar Bayesian kernel methods, like: batch RVM, fast RVM, variational RVM, and Gaussian processes on multistep ahead forecasting of time series.