Intelligent energy-efficient train trajectory optimization approach based on supervised reinforcement learning for urban rail transits

Abstract

Artificial intelligence of things (AIoT)-enabled intelligent automatic train operation (iATO) is an urgently needed technology to expand the capability of ATO in addressing the real-time responsiveness and dynamic online challenges to energy-efficient train trajectory optimization (TTO) and its associated ride-comfort, punctuality, and safety issues in modern urban rail transit networks. This paper proposes a three-step supervised reinforcement learning-based intelligent energy-efficient train trajectory optimization (SRL-IETTO) approach for iATO by hybrid-integrating deep reinforcement learning (DRL) and supervised learning. First, multiple objectives are formulated based on real-time train operation and systematically integrated into the RL algorithm by a binary function-based goal-directed reward design method. Second, an IETTO model is established to handle uncertain disturbances in real-time train operation and generate optimal energy-efficient train trajectories online by optimizing energy efficiency and receiving supervisory information from trajectories of pre-trained TTO models. Finally, numerical simulations are implemented to validate the effectiveness of the SRL-IETTO using in-service subway line data. The results demonstrate the superiority and improved energy saving of the proposed approach and confirm its adaptability to online trip time adjustments within the practical running time range under uncertain disturbances with less trip time error compared to other intelligent TTO algorithms.