An AR-Assisted deep learning-based approach for automatic inspection of aviation connectors

Abstract

The mismatched pins inspection of the complex aviation connector is a critical process to ensure the correct wiring harness assembly, of which the existing manual operation is error-prone and time-consuming. Aiming to fill this gap, this article proposes an augmented reality (AR)-assisted deep learning-based approach to tackle three major challenges in the aviation connector inspection, including the small pins detection, multipins sequencing, and mismatched pins visualization. First, the proposed spatial-attention pyramid network approach extracts the image features in multilayers and searches for their spatial relationships among the images. Second, based on the cluster-generation sequencing algorithm, these detected pins are clustered into annuluses of expected layers and numbered according to their polar angles. Finally, the AR glass as the inspection visualization platform, highlights the mismatched pins in the augmented interface to warn the operators automatically. Compared with the other existing methodologies, the experimental result shows that the proposed approach can achieve better performance accuracy and support the operator's inspection process efficiently.