Ramp metering based on adaptive critic designs

This paper presents the technique of adaptive critic designs (ACDs) for resolving some traffic control problems, such as urban traffic signal control, ramp metering, highway speed control and so on. ACDs are adaptive optimal methods, whose main idea is to approximate dynamic programming solutions by using a function approximation structure such as neural networks to approximate the cost function or its derivatives. They combine the positive features of both the traditional dynamic programming (e.g., optimality and feedback) and the neural networks (e.g., numerical and real-time properties). Due to these exciting merits, they are promising in handling the traffic control problems which have strong nonlinear, stochastic and real-time characteristics. In this paper, a close look at two typical adaptive critic designs is conducted. Specifically, we demonstrate the use of a specific adaptive critic design for the local on-ramp metering control problem. Simulation results indicate that the proposed controller has good performance