用语言分割图像中的纹理模式，而非完整物体。
Sub-Semantic Image Segmentation
- 将语言描述用于划分稳定视觉纹理区域，而非识别物体
- 在新数据集TextureADE上达到最优分割性能
- 适合关注细粒度图像理解与视觉语言对齐的研究者
图像分割通常基于视觉线索（如纹理）或对象语义（语义分割）。本文提出一种新的子语义图像分割范式，模糊了两者之间的界限。在子语义分割中，语言不用于命名完整物体，而是用于将图像划分为可被语言描述的稳定外观模式。为此，我们结合通用视觉-语言模型与SAM 3——一个支持提示的分割主干网络，其原生文本路径可将丰富描述定位为掩码。简单耦合存在多种失效问题，本文通过引入DETECTURE解决三个具体缺陷：纹理区域间语言泄露、分割主干内提示竞争、以及语言到掩码接口的语义扭曲。由于缺乏子语义分割数据集，我们构建了名为TextureADE的新数据集，基于ADE20K设计系统生成。在多个数据集上，DETECTURE在不同指标下均优于多个基线方法。代码已开源：https://github.com/Scientific-Computing-Lab/TextureDetecture。
原文摘要 · Abstract (English)
Images can be segmented based on visual cues (i.e., texture segmentation) or into objects (i.e., semantic segmentation). We propose a new category of sub-semantic image segmentation that blurs the line between the two. In sub-semantic image segmentation, language is not used to name whole objects. Instead, it is used to partition an image into stable appearance patterns that can be described by language. To do that, we couple a general-purpose vision-language model to SAM 3, a promptable segmentation backbone whose native text pathway can ground rich descriptions into masks. Simple coupling fails for a number of reasons that we identify in the paper, and we overcome them by introducing DETECTURE that resolves three concrete failure modes -- language leakage between texture regions, prompt competition inside the segmentation backbone, and semantic distortion at the language-to-mask interface. Since there is no dataset of sub-semantic image segmentation, we introduce one, termed TextureADE. The new dataset is derived from the ADE20K dataset using a system we designed. We compare DETECTURE to a number of baselines and find that it achieves the strongest performance on several datasets using different metrics. Code is available at https://github.com/Scientific-Computing-Lab/TextureDetecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。