A comparative study of urban traffic signal control with reinforcement learning and Adaptive Dynamic Programming

This paper proposes a new algorithm that employs Adaptive Dynamic Programming(ADP) to solve the distributed control problem of urban traffic with an infinite horizon. Urban traffic congestions lead to a lot of time consumption and exhaust emissions. So alleviating congested situation will have a good impact on both economy and environment. The signal control at urban intersections is an effective and most important way to reduce the traffic jams and collisions. A lot of control theories including traditional mathematical ways and modern artificial intelligent ways have been exploited. ADP is an effective and amiable intelligent control method. We proposed an algorithm to adjust the signal time plan at urban traffic intersections based on ADP theory. Simulations are taken under a microscopic traffic simulation software, TSIS(Traffic Software Integrated System). Several criteria named MOEs(Measures of Effectiveness) are collected to compare with the widely used pre-timed control, actuated control, also with a machine learning method Q-learning control. Results show that ADP control method have a better adaptability to the various traffic simulating real traffic flows.