On the Singular Value Manifold and Numerical Stabilization of Algorithms with Orthogonality Constraints

Recently, interest has risen in adaptive algorithms that implicitly impose orthogonality constraints on an adjustable matrix. In practice, parameter deviations from orthogonality can occur due to a chosen algorithm´s numerical implementation. This paper introduces the geometry of and adaptive algorithms for the singular value manifold to mitigate these numerical effects. Both gradient and Newton-based methods on the singular value manifold are derived. Applications to single-step and iterative orthogonalization reveal relationships between existing orthogonalization methods as well as novel, fast-converging approximate Newton procedures for this task. Simulations are used to explore their performances