arXiv:2412.11381cs.CVcs.AI2024-12被引 5

用GAN生成数据微调SAM,提升增材制造X射线图像分割精度。

Adapting Segment Anything Model (SAM) to Experimental Datasets via Fine-Tuning on GAN-based Simulation: A Case Study in Additive Manufacturing

  • 通过Conv-LoRa实现参数高效微调,降低计算开销。
  • 在真实工业数据上,微调后模型分割准确率显著提升。
  • 适合材料科学领域研究人员解决复杂结构图像分割问题。

工业X射线计算机断层扫描(XCT)是无损表征材料与制造部件的有力工具,常需结合先进图像分析与计算机视觉算法提取信息。传统计算机视觉模型因噪声、分辨率变化及复杂内部结构而表现不佳,尤其在科学成像中。以通用图像分割著称的分割一切模型(SAM)虽具潜力,但在材料科学领域应用仍有限。本文研究了SAM在增材制造部件XCT检测中的适用性,发现其在分布外数据、多类别分割和微调效率方面存在不足。为此,提出基于参数高效技术Conv-LoRa的微调策略,并利用生成对抗网络(GAN)生成数据增强训练,提升模型在复杂XCT数据上的分割性能。实验表明,针对特定材料数据集微调SAM能显著改善效果。然而,模型跨数据集泛化能力依然有限,凸显出对鲁棒、可扩展领域专用分割方案的迫切需求。

原文摘要 · Abstract (English)

Industrial X-ray computed tomography (XCT) is a powerful tool for non-destructive characterization of materials and manufactured components. XCT commonly accompanied by advanced image analysis and computer vision algorithms to extract relevant information from the images. Traditional computer vision models often struggle due to noise, resolution variability, and complex internal structures, particularly in scientific imaging applications. State-of-the-art foundational models, like the Segment Anything Model (SAM)-designed for general-purpose image segmentation-have revolutionized image segmentation across various domains, yet their application in specialized fields like materials science remains under-explored. In this work, we explore the application and limitations of SAM for industrial X-ray CT inspection of additive manufacturing components. We demonstrate that while SAM shows promise, it struggles with out-of-distribution data, multiclass segmentation, and computational efficiency during fine-tuning. To address these issues, we propose a fine-tuning strategy utilizing parameter-efficient techniques, specifically Conv-LoRa, to adapt SAM for material-specific datasets. Additionally, we leverage generative adversarial network (GAN)-generated data to enhance the training process and improve the model's segmentation performance on complex X-ray CT data. Our experimental results highlight the importance of tailored segmentation models for accurate inspection, showing that fine-tuning SAM on domain-specific scientific imaging data significantly improves performance. However, despite improvements, the model's ability to generalize across diverse datasets remains limited, highlighting the need for further research into robust, scalable solutions for domain-specific segmentation tasks.

图像分割GAN生成增材制造SAM微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。