Semi-Random Projection for Dimensionality Reduction and Extreme Learning Machine in High-Dimensional Space

Random Projection (RP) is a popular technique for dimensionality reduction because of its high computational efficiency. However, RP may not yield highly discriminative low-dimensional space to produce best pattern classification performance since the random transformation matrix of RP is independent of data. In this paper, we propose a Semi-Random Projection (SRP) framework, which takes the merit of random feature sampling of RP, but employs learning mechanism in the determination of the transformation matrix. One advantage of SRP is that it achieves a good balance between computational complexity and classification accuracy. Another advantage of SRP is that multiple SRP modules can be stacked to form a deep learning architecture for compact and robust feature learning. In addition, based on the insight on the relationship between RP and Extreme Learning Machine (ELM), the SRP is applied to ELM to derive Partially Connected ELM (PC-ELM). The hidden nodes of PC-ELM are more discriminative and hence a smaller number of nodes are needed. Experiments on two real-world text corpus, i.e., 20 Newsgroups and Farms Ads., verify the effectiveness and efficiency of the proposed SRP. Experimental results also show that PC-ELM outperforms ELM for high-dimensional data.