Application of stochastic control to analysis and optimization of TCP

The article considers the optimal stochastic control approach to the analysis of TCP. Generally the analysis of TCP schemes is based on so-called fluid models which provide the asymptotic behavior of the queuing system where the number of jobs is huge. At this level of consideration only the asymptotic results could be obtained and the real performance of existing protocols is still unclear particularly if there are the seasonal changes and congestions. Meanwhile, the models of controllable Markov chains (CMC) are more appropriate to the analysis of control of flows in the Internet that has been observed by many authors long ago. The principal difficulty of the CMC application is the high dimension particularly for connected controllable Markov chains (CCMC). But nowadays this problem is less important due to the development of multiprocessor supercomputers that make the numerical solution of the optimal control problems for CMC more achievable. Models of CCMC arise in queuing systems with many service lines where some idle lines may be used to avoid the congestion if the principal lines have been subjected the huge workload. Here we suggest the tensor form of such CMC description and give the dynamic programming equation in corresponding tensor form. As examples we consider the results of CMC approach to control the system with one service line, system with two service lines, namely the main and reserve ones, having different service rates and the cost of service, and to the system with incomplete information about the router state.