Event-Based Stochastic Learning and Optimization

Summary form only given. In many modern engineering systems, control actions are taken only when some events occur. In networking admission control, an action (accept or reject) is taken only when a new packet arrives; in power control of wireless communication where a mobile device travels among regions with different transmission environments, a decision (transmission rate) is made only when the mobile device enters a new region; in an inventory problem with delayed information, decision depends on the partially observed information which can also be viewed as events; in a flexible manufacturing system, actions (which work piece to process next) are taken only when a work piece is completed. The traditional Markov decision process (MDP) model does not fit these problems well and may unnecessarily suffer from the curse-of-dimensionality issue. Performance optimization of such problems can be solved by an event-based approach. This approach involves three main topics: 1) the formulation of events and event-based policies; 2) the sensitivity based view for performance optimization; and 3) computational savings, learning and on-line implementation. The paper presents a brief introduction to the above topics and discusses pros and cons of this new approach