Study on the effect of learning parameters on decision boundary making algorithm

The purpose of our study is to induce compact and high performance machine learning models. In our earlier study, we proposed a decision boundary making (DBM) algorithm. The main philosophy of the DBM algorithm is to reconstruct a high performance model with much smaller cost. In our study, we use support vector machine as a high performance model, and a multilayer neural network, i.e., multilayer perceptron (MLP), as the small model. Experimental results obtained so far show that high performance and compact MLPs can be obtained using DBM. However, there are several parameters of DBM that need to be adjusted appropriately in order to achieve better performance. In this paper, we investigate the effect of parameter N, which is the number of newly generated data, on the performance of obtained MLPs. We discuss the issue that how many new data we should generate to obtain a better performance of DBM. We also investigate the effect of outliers on the performance of the obtained MLPs. Outliers are generally known to be harmful for pattern recognition. Our experimental results show, however, that for some databases, outliers can be useful for obtaining high performance MLPs.