Studying the possibility of peaking phenomenon in linear support vector machines with non-separable data

Typically, it is common to observe peaking phenomenon in the classification error when the feature size increases. In this paper, we study linear support vector machine classifiers where the data is non-separable. A simulation based on synthetic data is implemented to study the possibility of observing peaking phenomenon. However, no peaking in the expected true error is observed. We also present the performance of three different error estimators as a function of feature and sample size. Based on our study, one might conclude that when using linear support vector machines, the size of feature set can increase safely.