AviationCopilot : building a reliable LLM-based Aviation Copilot inspired by human pilot training

Abstract

Modern pilots routinely face high cognitive loads during complex flight operations. Although large language models (LLMs) demonstrate exceptional natural language understanding and exhibit tremendous potential as copilots, there is a notable gap in LLMs specifically designed to handle the knowledge-intensive tasks required of pilots. Inspired by pilots’ learning and manual retrieval patterns, we introduce AviationCopilot—a novel framework that efficiently injects both aviation knowledge content and knowledge structure into LLMs. Specifically, we employ differentiated data fusion and generalization strategies for two training stages including continual pre-training and instruction tuning. This approach equips the model with enhanced domain-specific knowledge retention and instruction-following capabilities, akin to human pilots. During inference, AviationCopilot activates its knowledge structure memory to adaptively retrieve comprehensive context, improving factual accuracy. To evaluate effectiveness, we construct a comprehensive benchmark named OpenAviation featuring both LLM-synthesized and expert-designed questions. Experimental results show that models with fewer than two billion parameters, trained with the AviationCopilot framework, consistently outperform strong LLM baselines, including those utilizing Retrieval-Augmented Generation (RAG). Additionally, AviationCopilot enhances structured aviation understanding and enables LLMs to serve as retrievers for improving other models, supporting more reliable AI copilots.