让分割模型更关注纹理而非形状，提升医学图像等场景的精度
TextureSAM: Towards a Texture Aware Foundation Model for Segmentation
- 通过纹理增强技术微调基础模型，引导其重视纹理特征
- 在真实与合成纹理数据集上分别优于SAM-2 0.2和0.18 mIoU
- 适合纹理主导的医学、遥感等领域的图像分割任务
分割一切模型（SAM）在多种数据集上的物体分割任务中取得了显著成功。然而，这些模型主要在大规模语义分割数据集上训练，导致其对图像中的物体形状敏感，而忽视纹理线索。这一缺陷在医学影像、材料分类和遥感等领域尤为关键，因为物体边界常由纹理变化定义。本文研究了SAM对语义的偏好与对纹理的忽视，并提出新的纹理感知基础模型TextureSAM，其在纹理主导场景中表现更优。为此，我们采用一种新颖的微调方法，通过纹理增强技术逐步修改训练图像，强化纹理特征。基于对ADE20K数据集的新型纹理交替处理，引导TextureSAM优先关注纹理定义区域，从而缓解原始SAM中存在的形状偏差。大量实验表明，TextureSAM在自然纹理数据集上比SAM-2高0.2 mIoU，在合成纹理数据集上高0.18 mIoU。代码与增强数据集将公开发布。
原文摘要 · Abstract (English)
Segment Anything Models (SAM) have achieved remarkable success in object segmentation tasks across diverse datasets. However, these models are predominantly trained on large-scale semantic segmentation datasets, which introduce a bias toward object shape rather than texture cues in the image. This limitation is critical in domains such as medical imaging, material classification, and remote sensing, where texture changes define object boundaries. In this study, we investigate SAM's bias toward semantics over textures and introduce a new texture-aware foundation model, TextureSAM, which performs superior segmentation in texture-dominant scenarios. To achieve this, we employ a novel fine-tuning approach that incorporates texture augmentation techniques, incrementally modifying training images to emphasize texture features. By leveraging a novel texture-alternation of the ADE20K dataset, we guide TextureSAM to prioritize texture-defined regions, thereby mitigating the inherent shape bias present in the original SAM model. Our extensive experiments demonstrate that TextureSAM significantly outperforms SAM-2 on both natural (+0.2 mIoU) and synthetic (+0.18 mIoU) texture-based segmentation datasets. The code and texture-augmented dataset will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。