Tying rotations of covariance matrices via riemannian subspace clustering

The use of full covariance matrices in acoustic modeling is getting popular, but its huge computational burden in likelihood calculation is a major issue. Semi-tied covariance matrices are commonly used to speed-up the computation, where global or phone-based tying of transforms, or “rotations”, is usually used. However, such tyings are heuristic, and not necessarily optimal. In this paper, we propose a Riemannian-geometric approach to optimally tying rotations of covariance matrices. We first introduce a tangent space of the Riemannian manifold of covariance matrices, which has an excellent distance for measuring dissimilarity between covariance matrices. We then show that covariance matrices having the same rotation to each other lie on the same subspace in the tangent space. Exploiting this property, we fit subspaces to samples (covariances) in the tangent space for finding out clusters of samples that have similar rotations, and tie them together. By doing so, an optimal tying that minimizes the sum of “distortions” of covariance matrices can be found. Experimental results on the Wall Street Journal corpus show a superior performance of the proposed tying over the conventional ones.