Deep Twin Support Vector Machine

We propose a novel machine learning model for classification problems, Deep Twin Support Vector Machine (DTWSVM), which combines TWSVM with deep learning ideas. TWSVM is a successful algorithm for classification problems which seeks two nonparallel hyper planes to make each hyper plane close to one class and far from the other as much as possible. And Deep Learning (DL) models have shown good ability in feature extraction and dimension reduction by constructing multi-layer network. Since the feature extraction in DL can reduce feature dimension while maintain the main information of original inputs, we consider constructing a three layer network which contains input layer, hidden layer and output layer. We use two TWSVMs in the hidden layer to extract features based on the projection principle which is derived from Multi-Layer Perceptron (MLP). The two TWSVMs will get four hyper planes by solving four convex quadratic programs. A new dataset which consists of the extracted features with four feature dimensions is produced from the hidden layer and then we can input it to the main TWSVM of the output layer to make final prediction. Similar as TWSVM, we design linear DTWSVM and nonlinear DTWSVM which have been proved to be very effective in classification problems. In numerical experiments, we have obtained 100% prediction accuracy for several datasets which is state-of-the-art performance absolutely!