A transformer–VAE approach for detecting ship trajectory anomalies in cross-sea bridge areas

Abstract

Abnormal ship navigation behaviors in cross-sea bridge waters pose significant threats to maritime safety, creating a critical need for accurate anomaly detection methods. Ship AIS trajectory data contain complex temporal features but often lack explicit labels. Most existing anomaly detection methods heavily rely on labeled or semi-supervised data, thus limiting their applicability in scenarios involving completely unlabeled ship trajectory data. Furthermore, these methods struggle to capture long-term temporal dependencies inherent in trajectory data. To address these limitations, this study proposes an unsupervised trajectory anomaly detection model combining a transformer architecture with a variational autoencoder (transformer–VAE). By training on large volumes of unlabeled normal trajectory data, the transformer–VAE employs a multi-head self-attention mechanism to model both local and global temporal relationships within the latent feature space. This approach significantly enhances the model’s ability to learn and reconstruct normal trajectory patterns, with reconstruction errors serving as the criterion for anomaly detection. Experimental results show that the transformer–VAE outperforms conventional VAE and LSTM–VAE in reconstruction accuracy and achieves better detection balance and robustness compared to LSTM–-VAE and transformer–GAN in anomaly detection. The model effectively identifies abnormal behaviors such as sudden changes in speed, heading, and trajectory deviation under fully unsupervised conditions. Preliminary experiments using the POT method validate the feasibility of dynamic thresholding, enhancing the model’s adaptability in complex maritime environments. Overall, the proposed approach enables early identification and proactive warning of potential risks, contributing to improved maritime traffic safety.