Multi-agent reinforcement learning for cooperative transit signal priority to promote headway adherence

Abstract

Headway regularity is an essential indicator of transit reliability, directly influencing passenger waiting time and transit service quality. In this paper, we employ multi-agent reinforcement learning (MARL) to develop a Cooperative Transit signal priority strategy with Variable phase for Headway adherence (CTVH) under a multi-intersection network. Each signalized intersection is controlled by an RL agent, which determines the next step's signal, adapting to real-time traffic dynamics of transits and non-transits and promoting transit headway adherence. The proposed approach considers four critical aspects, i.e., complicated states with multiple conflicting bus requests, rational actions constrained by domain knowledge, comprehensive rewards balancing buses and cars, and a collaborative training scheme among agents. They are correspondingly addressed by proper state representation with estimated bus headway deviations, irrational actions masking, reward functions formulated by general traffic queue and transit headway deviation, and appropriate MARL approach with synchronous action processing. Our method also takes into account the phase transition loss by setting yellow and all-red time. Simulation results compared with the coordinated fixed-time signal (CFT) and bus holding (BH) strategy verify the merits of the proposed method in terms of improvements in transit headway adherence and influence on general traffic. Based on the results, we further discuss the BH method's limitations due to bus bay length and various holding lines and the CTVH method's benefits in the three-intersection environment and the entire-line network. The proposed method has a promising application in practice to improve transit reliability.