Novel Newton algorithms for the Hermitian eigenvalue problem

We present three related algorithms for iteratively computing all the eigenvectors of a Hermitian matrix. The algorithms are based on the idea of applying Newton updates to individual eigenvectors at each iteration. The advantage of these Newton updates is that they have a cubic rate of convergence. The difference between the algorithms is how they prevent the individual updates from converging to the same eigenvector. The first algorithm finds the eigenvectors sequentially, and uses a novel form of deflation in order to ensure all the eigenvectors are found. Rather than modify the matrix directly, which introduces large errors if the matrix is ill-conditioned, deflation is achieved by restricting the Newton updates to lie in a subspace orthogonal to all previously found eigenvectors. The other algorithms estimate all the eigenvectors at once. At each iteration, they sweep through all the estimates, performing a Newton update on each estimate once per sweep. Orthogonality is maintained by explicit re-orthogonalisation after each update, which also serves to improve the asymptotic rate of convergence of the algorithms.