Perturbation analysis of feedback-controlled stochastic flow systems

Stochastic flow systems arise naturally or as abstractions of discrete-event systems (DESs), referred to as stochastic flow models (SFMs). In this paper, we consider such systems operating with a feedback control mechanism, building on earlier work that has studied such SFMs without any feedback. Using infinitesimal perturbation analysis, we derive gradient estimators for loss and workload related performance metrics with respect to threshold parameters used for buffer control. These estimators are shown to be unbiased. They are also shown to depend only on data observable from a sample path of the actual DES. This renders them computable in on-line environments and easily implementable for control and performance optimization purposes. In the case of linear feedback, we further show that the estimators are nonparametric. Finally, we illustrate the use of these estimators in network control by combining them with standard gradient-based stochastic optimization schemes and providing several simulation-based examples.