Learning with Mean-Variance Filtering, SVM and Gradient-based Optimization

We consider several models, which employ gradient-based method as a core optimization tool. Experimental results were obtained in a real time environment during WCCI-2006 Performance Prediction Challenge. None of the models were proved to be absolutely best against all five datasets. Furthermore, we can exploit the actual difference between different models and create an ensemble system as a complex of the base models where the balances may be regulated using special parameters or confidence levels. Overfitting is a usual problem in the situation when dimension is comparable with the sample size or even higher. Using mean-variance filtering we can reduce the difference between training and test results significantly considering some features as a noise.