Priority-driven reinforcement learning for multi-aircraft trajectory optimisation under dynamic weather hazards

Abstract

Reinforcement Learning (RL) has emerged as a state-of-the-art technique for addressing challenges in air traffic control, and weather hazards and flight procedures can contribute to information biases when applying RL to real-world scenarios. This research focuses on the 3D Multi-Aircraft Trajectory Optimisation (3D-MATO) problem under dynamic weather hazards within the Terminal Manoeuvring Area and addresses the aforementioned concerns. We propose an integrated RL-based algorithm incorporating weather avoidance and quick conflict resolution. Given observed weather radar in Flight Information Regions (FIRs), we introduce the Dynamic Fast Marching Method (DFMM) algorithm to reroute flight paths at smaller time intervals, ensuring safer navigation around hazardous regions. To enhance decision-making quality, we develop a Quickest Priority-based Conflict Resolution (QPCR) strategy, which optimises approach sequences and refines available action choices. The RL agent is trained using a Deep Deterministic Policy Gradient (DDPG) framework, and further enhanced with a self-attention mechanism. A numerical study modelled the real-world approach procedures at Hong Kong International Airport involving varying numbers of approach aircraft under dynamic weather hazards. Results demonstrate the high efficiency and effectiveness of the proposed algorithm under traffic mix and weather conditions, highlighting the contributions of its key strategies and individual components.