Beyond the gaze : peripheral vision-aware visual detection failures recognition through LLM-based fixation coordinate-sensitive analysis

Abstract

Visual detection failures are a critical challenge in air traffic control (ATC), where undetected alerts can compromise operational safety and decision-making. Previous studies have primarily assessed detection failures through target fixation patterns, yet this method struggles to identify the more complex “look-but-fail-to-see” and “see-without-looking” scenarios. This underscores the necessity of exploring peripheral vision mechanisms, where dynamic tracking trajectories could better capture the scope of visual attention. Therefore, this study proposes a classification framework for visual detection by integrating peripheral vision tracking and human attentional states, including detection failures such as peripheral vision neglect and look-but-fail-to-see errors. A hierarchical detection failure recognition framework specific to the ATC settings is further developed and validated through an ATC simulation experiment. The framework first employs an Adaptive Symbolic Alert Detection method to identify and annotate ATC-specific alert regions with spatiotemporal uncertainty (achieving 95.24% precision), followed by LLM-based evaluation of operators’ visual attention to these regions to intelligently assign classification labels. Additionally, we introduce a fixation coordinate-sensitive multi-domain feature set that captures spatiotemporal and frequency-domain characteristics across detection types, achieving 93.13% four-class classification accuracy, outperforming traditional feature sets (83.69%) and both single-and dual-domain features (ranging from 76.82% to 90.11% accuracy). These findings demonstrate that our framework effectively captures a broader and structured range of visual detection failures, providing critical insights to improve the reliability of alert detection in ATC and the design of an intelligent human-centered ATC support system.