Tree-Structured Learning of Multi-class SVMs with Triple Learning Units

To reduce the computational complexity of multi-class Support Vector Machines (SVM), this paper presents a multi-class algorithm in which a triple classifier is included as a second learning unit. This triple learning unit is a regression model for three classes and is based on Least-Squares SVMs (LS-SVMs). To train the triple learning unit, binary target values are first expanded with a third optional output, then an advanced LS-SVM algorithm is used to guarantee the sparseness of the solution. An ensemble of all learning units are placed at nodes of a Directed Decision Tree (DDT), leading to proposal of a Directed Decision Tree SVM (DDTSVM). DDTSVMs can improve the learning efficiency in classifying unlabelled data, a drawback for both 1-v-r and 1-v-1methods. Empirical studies show that the proposed DDTSVM achieves excellent classification accuracy in comparison with the 1-v-1 method.