A model-free approach to optimal signal light timing for system-wide traffic control

A long-standing problem in traffic engineering is to optimize the flow of vehicles through a given road network. Improving the timing of the traffic signals at intersections in the network is generally the most powerful and cost-effective means of achieving this goal. However, because of the many complex aspects of a traffic system-human behavioral considerations, vehicle flow interactions within the network, weather effects, traffic accidents, long-term (e.g., seasonal) variation, etc.-it has been notoriously difficult to determine the optimal signal light timing. This is especially the case on a system-wide (multiple intersection) basis. Much of this difficulty has stemmed from the need to build extremely complex open-loop models of the traffic dynamics as a component of the control strategy. This paper presents a fundamentally different approach for optimal light timing that eliminates the need for such an open-loop model. The approach is based on a neural network (or other function approximator) serving as the basis for the control law, with the weight estimation occurring in closed-loop mode via the simultaneous perturbation stochastic approximation (SPSA) algorithm. Since the SPSA algorithm requires only loss function measurements (no gradients of the loss function), there is no open-loop model required for the weight estimation. The approach is illustrated by simulation on a six-intersection network with moderate congestion and stochastic, nonlinear effects