arXiv:2606.14755cs.CVcs.AI2026-06

探究SAM模型中纹理信息的留存位置与利用方式

Where Does Texture Evidence Live in SAM? Features, Proposal Masks, and Texture Segmentation

论文配图:Where Does Texture Evidence Live in SAM? Features, Proposal Masks, and Texture Segmentation
图 1 · 摘自论文原文
  • 分析冻结SAM的多尺度特征与候选掩码库中的纹理证据
  • 发现粗糙特征保留纹理结构,候选掩码常含纹理对齐片段
  • 适用于研究视觉基础模型对非物体定义区域的表征能力

纹理分割对基础分割模型构成挑战,因有意义区域由材质或重复外观定义而非对象身份。标准分割任意图像时,段落分割模型(SAM)通常表现不佳,但这种失败原因模糊:纹理证据可能缺失、未出现在候选掩码池中,或虽存在却因以对象为中心的读出机制被错误选择或组装。本文研究在不微调或重训练的情况下,冻结的SAM模型中是否已存留纹理相关证据。考察两个冻结的证据空间:使用最小聚类读出探查的多尺度特征,以及作为监督整合读出证据的自动候选掩码池。在RWTD、STLD、ADE20K筛选的精简裁剪补集和ControlNet拼接的PTD桥梁数据集上,冻结的SAM默认并非纹理分割器,但其失败并非简单的纹理盲视。粗粒度特征保留了纹理组织结构,候选掩码池中常包含对齐纹理的掩码或片段。自然场景更需对片段进行组装与确认,而更干净的合成场景则往往只需选择已连贯的候选掩码。因此,默认掩码失败应分解为表示证据、候选池支持、读出不匹配及确认失败四类。

原文摘要 · Abstract (English)

Texture segmentation stresses foundation segmentation because meaningful regions are defined by material or repeated appearance rather than object identity. Segment Anything Models (SAMs) often fail by default on such texture-defined partitions, but this failure is ambiguous: the texture evidence may be absent, missing from the proposal bank, or present but selected or assembled incorrectly by an object-centric readout. We ask what texture-relevant evidence is already preserved in frozen SAM before adaptation. We study two frozen evidence spaces: multiscale features, probed with a minimal clustering readout, and the automatic proposal bank, treated as evidence for a supervised consolidation readout. SAM is frozen throughout; we do not fine-tune the backbone or retrain the proposal generator. Across RWTD, STLD, an ADE20K-selected refined-crop complement, and a ControlNet-stitched PTD bridge archive, frozen SAM is not a texture segmenter by default, but its failures are not simple texture blindness. Coarse frozen features preserve texture organization, and proposal banks often contain texture-aligned masks or fragments. Natural scenes more often require assembly and commitment over fragments, while cleaner synthetic cases more often reduce to selecting an already coherent proposal. Default mask failure should therefore be decomposed into representation evidence, proposal-bank support, readout mismatch, and commitment failure.

纹理分割SAM特征表征零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。