A bias-variance dilemma in Joint Diagonalization and Blind Source Separation

We identify and explain a bias-variance dilemma which exists in the problem of approximate matrix joint diagonalization (JD) as well as in many related blind source separation (BSS) problems. We consider solving a blind identification problem based on JD, where at least one of the matrices under JD is positive definite. We then compare two methods to solve the problem: The first method consists of the so-called Hard-Whitening (HW) followed by orthogonal JD (OJD), and the second method is based on non-orthogonal JD (NOJD). We identify a bias-variance trade-off in this problem, and argue that there is a region depending on the noise level, the number of sources and the number of (statistics) matrices used in the JD process, where the method based on OJD can have less estimation error than the one based on NOJD, while the former always has higher estimation bias than the latter. Simulations support the arguments presented. We also report a constraint proposed in the literature which might be helpful in finding a good trade-off point between bias and variance.