An optimization algorithm with probabilistic estimation

The authors present a stochastic optimization algorithm based on the idea of the gradient method which incorporates a novel adaptive-precision technique. Unlike recent methods, the proposed algorithm adaptively selects the precision without any need for prior knowledge on the speed of convergence of the generated sequence. The algorithm can avoid increasing the estimation precision unnecessarily, yet it retains its favorable convergence properties. In fact, it tries to maintain a nice balance between the requirements for computational accuracy and those for computational expediency. The authors present two types of convergence results delineating under what assumptions various kinds of convergence can be obtained for the proposed algorithm. They also present a parallel version of the proposed algorithm