Multimodal fusion for underwater localization through hierarchical uncertainty awareness

Abstract

Traditional vehicle localization methods, such as visual simultaneous localization and mapping (SLAM), often struggle underwater, especially under conditions like motion blur and low light. To address these challenges, this article presents a novel multimodal fusion system for underwater localization that integrates hierarchical uncertainty awareness (UA) into the acoustic/visual/inertial/pressure (AVIP) fusion framework, termed UA-AVIP, to achieve robust and accurate localization. The Doppler velocity log (DVL) is introduced for reliable scale information, and a pressure sensor (PS) for water-depth measurements. A dual-framework approach is introduced to parallelize the fusion of acoustic and pressure data in an extended Kalman filter (EKF) framework with the integration of visual and inertial measurements via a visual-inertial system. The final estimates are subsequently unified through factor graph optimization (FGO). Furthermore, a hierarchical UA strategy is developed to effectively manage uncertainties at multiple levels within the dual-framework fusion. At the measurement level, noise variance is adaptively estimated through spectrum analysis. At the module level, a deep learning-assisted submodel enhances acoustic and pressure estimation. At the system level, an adaptive weighting strategy within the factor graph addresses dynamic uncertainties. Both self-collected and open-source datasets in the pool, lakes, and oceans under distinct scenarios are used to train and validate the effectiveness and superiority of the proposed UA-AVIP system. The results underscore the critical importance of UA in underwater localization. The source code is available at https://github.com/HKPolyU-UAV/UA-AVIP