Self-evolving prompting segment anything model for crack segmentation through data-driven cyclic conversations

Abstract

Recent advancements in large-scale segmentation models, notably the SAM (Segment Anything Model), have significantly impacted the domain of image segmentation. Despite these advancements, crack segmentation remains a challenging task due to the irregular and unstructured characteristics of crack patterns, which often necessitate extensive manual prompting. To address this issue, we introduce a novel self-autonomous prompting framework for SAM, named SepSAM, specifically designed for crack segmentation applications. Rather than relying on a large model with substantial training demands, SepSAM novelly employs a lightweight crack detection model as a prompting agent, allowing the SAM to effectively interpret crack patterns. Comparative analyses reveal that the proposed method achieves notable improvements in recall (approximately 5 % for cracks wider than 20 pixels) and surpasses existing similar methods in overall performance metrics. Comprehensive ablation studies confirm the efficacy of the proposed adaptation. With a total trainable parameter size of less than 7 MB, SepSAM is particularly well-suited for on-site bridge and pavement inspection deployments.