What Can Make Joint Diagonalization Difficult?

We show that the issues of uniqueness and noise sensitivity in the problem of matrix joint diagonalization are closely related. We address other factors important in noise sensitivity. We distinguish between orthogonal and non-orthogonal joint diagonalization and argue that the latter can be more difficult than the former. Our analysis is based on the perturbation analysis of the stationary points of certain flows for joint diagonalization. Numerical experiments support the derived results.