Adaptive cyclic and randomized coordinate descent for the sparse total least squares problem

Coordinate descent (CD) is a simple and general optimization technique. We use it to solve the sparse total least squares problem in an adaptive manner, working on the l₁-regularized Rayleigh quotient function. We propose two algorithmic approaches for choosing the coordinates: cyclic and randomized. In both cases, the number of CD steps per time instant is a parameter that can serve as a trade-off between complexity and performance. We present numerical experiments showing that the proposed algorithms can approach stationary error near that of the oracle. The randomized algorithm is slightly better than the cyclic one with respect to convergence speed.