Constrained and Preconditioned Stochastic Gradient Method

We consider stochastic approximations that arise from such applications as data communications and image processing. We demonstrate why constraints are needed in a stochastic approximation and how a constrained approximation can be incorporated into a preconditioning technique to derive the preconditioned stochastic gradient method (PSGM). We perform convergence analysis to show that the PSGM converges to the theoretical best approximation under some simple assumptions on the preconditioner and on the independence of samples drawn from a stochastic process. Simulation results are presented to demonstrate the effectiveness of the constrained and preconditioned stochastic gradient method.