On the theoretical and computational analysis between Trace Ratio LDA and null-space LDA

Linear Discriminant Analysis (LDA) is a well-known dimensionality reduction algorithm for pattern recognition and machine learning. And Trace Ratio LDA (TR-LDA) and Null-space LDA (NLDA) are two popular variants of LDA. Both NLDA and TR-LDA can result in orthogonal transformations. However, they applied different schemes in deriving the optimal transformation. NLDA computes an orthogonal transformation in the null space of the within-class scatter matrix, while TRLDA computes an orthogonal transformation by an iterative procedure. In this paper, by using the trace difference problem as a bridge, we show that the above two algorithms can be equivalent when confronts with singularity problem. In addition, extensive simulations were conducted based on several datasets. Both theoretical analysis and simulation results confirm the equivalent relationship.