Performance Comparison between Queueing Theoretical Optimality and Q-Learning Approach for Intersection Traffic Signal Control

This paper proposes the performance comparison for optimal traffic signal controls based on the following two frameworks: M/M/1 and D/D/1 queueing models, and Q-learning approach. Firstly, using the M/M/1 and D/D/1 models, the optimal split derivation has been obtained to minimise the mean waiting time of an intersection. Additionally, the Q-learning framework has been proposed in conjunction with the use of the macroscopic cell transmission model (CTM) to update the vehicle state dynamics upon Q-learning actions. The two approaches have been compared in terms of the network throughput and the average vehicle delay per completed trip in nine scenarios. The simulation results from the microscopic AIMSUN traffic simulator show that the Q-learning approach can greatly improve the intersection throughput and can significantly reduce the average vehicle delay per completed trip with the respective M/M/1 and D/D/1 approaches.