Adaptive mirror descent algorithm for the minimization of expected cumulative losses driven by a renewal process

The problem considered in this paper is the minimization of expected cumulative losses in a stochastic system. The losses over time horizon are formed by the values of an unknown loss function at the consecutive jump times of a renewal process. The loss is assumed to be a convex function of a vector parameter, and the only available information is represented by an oracle which provides stochastic sub-gradients of the loss function. The control objective is to minimize the expected cumulative loss over a given convex compact set. We propose an adaptive mirror descent algorithm and prove an explicit upper bound for the related regret, which is the difference between the expected cumulative losses and the minimum. Finally, to exemplify the efficiency of the method, we consider the problem of minimization of the expected cumulative losses over the standard simplex by handling a stream of losses arriving by the Erlang process, and we discuss the simulation results.