Adaptive nonstationary regression analysis

The problem of finding the most appropriate subset of features or regressors is the generic challenge of Machine Learning problems like regression estimation or pattern recognition. We consider the problem of time-varying regression estimation, which implies also the inevitable necessity to choose the individual appropriate levels of model volatility in each of regressors, ranging from the full stationarity of instant models to their absolute independence in time. The problem is considered from the Bayesian point of view as that of estimating the sequence of regression coefficients associated with the hidden vector state of a stochastic linear dynamic system, whose a priori model includes parameters responsible for both the size of the subset of active regressors and the time-volatility factors of regression coefficients at them. The proposed technique of adaptive time varying regression estimation is built as that of estimating both the state and parameters of the hidden state-space model.