Formulation and integration of learning differential equations on the stiefel manifold

This letter aims at illustrating the relevance of numerical integration of learning differential equations on differential manifolds. In particular, the task of learning with orthonormality constraints is dealt with, which is naturally formulated as an optimization task with the compact Stiefel manifold as neural parameter space. Intrinsic properties of the derived learning algorithms, such as stability and constraints preservation, are illustrated through experiments on minor and independent component analysis (ICA).