将物体描述拆解为概念与属性，提升细粒度开放词汇分割的泛化能力。
Decomposed Vision-Language Alignment for Fine-Grained Open-Vocabulary Segmentation
- 将文本提示分解为概念和属性独立处理，实现语义单元分开展示
- 在特征层通过门控注意力融合属性信息，增强组合语义表达能力
- 支持现有模型无缝接入，显著提升未见属性-类别组合的识别效果
开放词汇分割模型在面对未见过的物体类别与属性组合时表现不佳，因细粒度描述通常以整体句子形式编码，混淆了多个语义单元。本文提出一种分解式视觉-语言对齐框架，将文本提示显式分解为概念标记和多个属性标记，使每个语义单元可独立进行跨模态交互。在特征层面，引入特征门控交叉注意力模块，生成属性特异的门控图，以乘法方式融合信息，有效强化组合语义。在评分层面，采用对数空间聚合各标记相似度，实现稳定且可解释的组合匹配。该方法可无缝集成至现有基于Transformer的分割架构，在细粒度开放词汇分割基准上显著提升对未见属性-类别组合的泛化性能。
原文摘要 · Abstract (English)
Open-vocabulary segmentation models often struggle to generalize to unseen combinations of object categories and attributes, because fine-grained descriptions are typically encoded as holistic sentences that entangle multiple semantic units. We propose a Decomposed Vision-Language Alignment framework that explicitly factorizes textual prompts into a concept token and multiple attribute tokens, enabling separate cross-modal interactions for each semantic unit. At the feature level, we introduce a Feature-Gated Cross-Attention module that generates attribute-specific gating maps to fuse information in a multiplicative manner, effectively enforcing compositional semantics. At the scoring level, per-token similarities are aggregated in log-space, producing a stable and interpretable compositional matching. The method can be seamlessly integrated into existing transformer-based segmentation architectures and significantly improves generalization to unseen attribute-category compositions in fine-grained open-vocabulary segmentation benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。