Digital Modulation Classifier with Rejection Ability via Greedy Convexhull Learning and Alternative Convexhull Shrinkage in Feature Space

Automatic modulation classification is a key technique in wireless communication systems. In practical scenario, the modulation format of a received signal may be new or unknown to classifiers. However, most of existing classifiers compulsively classify the received signal as one of the candidate modulation formats. The rejection ability of a classifier to unknown modulation formats is important in applications. In this paper, a fourth-order cumulant-based classifier with rejection ability is proposed to classify digital modulation formats under additive white Gaussian noise channel. The two fourth-order cumulants are used as the feature vector. The classification with rejection ability boils down to the segmentation problem of the two-dimensional feature space with rejection region. A two-stage optimization is proposed to attain the suboptimal solution of the problem. The greedy convexhull learning algorithm is used to determine the primary decision regions of all the candidate modulation formats from training sets. The primary decision regions have rejection ability but are not always mutually separate. The alternative convexhull shrinkage is presented to separate the primary decision regions at small loss in probability of correct classification (PCC). The proposed classifier is compared with the cumulant-based hierarchical classifier and the K-S classifiers. The results show that besides desired rejection ability it is competitive with these classifiers in PCC.