Markovian real-time adaptive control of signal systems

An approach to real-time control of a network of signalized intersections is proposed based on a discrete time, stationary, Markov control model (also known as Markov decision process or Markov dynamic programming). The approach incorporates microscopic simulation of actuated controller output signals in response to probabilistic forecasts of individual vehicle actuations at downstream inductance loop detectors derived from a macroscopic link transfer function. An Artificial Neural Network representation of vehicle delay estimations is proposed and tested for approximate real-time evaluation of potential traffic signal transitions at three-second evaluation intervals. A series of off-line tests of the developed procedures are applied to a simplified network of five intersections: these tests provide promising indications of this approach.