Efficient domain knowledge injection for bridging the gap between generalized large vision models and specialized fabric defect tasks

Abstract

The scarcity of high-quality annotated data poses a significant challenge to the application of deep learning in fabric defect tasks, limiting the generalization and segmentation performance of existing models and impeding their capability to address the complexity of various fabric types and defects. To overcome these obstacles, this study introduces an innovative method to infuse specialized knowledge of fabric defects into the Segment Anything Model (SAM), a large-scale visual model. By introducing and training a unique set of fabric defect-related parameters, this approach seamlessly integrates domain-specific knowledge into SAM without the need for extensive modifications to the preexisting model parameters. The revamped SAM model leverages generalized image understanding learned from large-scale natural image datasets while incorporating fabric defect-specific knowledge, ensuring its proficiency in fabric defect segmentation tasks. The experimental results reveal a significant improvement in the model’s segmentation performance, attributable to this novel amalgamation of generic and fabric-specific knowledge. When benchmarking against popular existing segmentation models across three datasets, our proposed model demonstrates a substantial leap in performance. Its impressive results in cross-dataset comparisons and few-shot learning experiments further demonstrate its potential for practical applications in textile quality control. 高质量带注释数据的稀缺性对深度学习在织物缺陷任务中的应用提出了重大挑战，限制了现有模型的泛化和分割性能，并阻碍了它们解决各种织物类型和缺陷复杂性的能力. 为了克服这些障碍，本研究引入了一种创新方法，将织物缺陷的专业知识注入到大规模视觉模型分段任意模型（SAM）中. 通过引入和训练一组独特的织物缺陷相关参数，这种方法将特定领域的知识无缝集成到SAM中，而不需要对预先存在的模型参数进行大量修改. 改进后的SAM模型利用从大规模自然图像数据集中学习到的广义图像理解，同时结合织物缺陷特定知识，确保其在织物缺陷分割任务中的熟练程度. 实验结果表明，由于这种通用知识和织物特定知识的新颖融合，模型的分割性能得到了显著提高. 当在三个数据集上对流行的现有分割模型进行基准测试时，我们提出的模型在性能上实现了巨大的飞跃. 它在跨数据集比较和少数镜头学习实验中的令人印象深刻的结果进一步证明了它在纺织品质量控制中的实际应用潜力.