用双路径对齐提升开放词汇语义分割的准确性
Semantic Alignment in Hyperbolic Space for Open-Vocabulary Semantic Segmentation
- 分离层次与语义对齐，在庞加莱球中分别优化半径与角度
- 在PASCAL-Context和LVIS上达到新最好结果，性能超越现有方法
- 适合做开放词汇分割、需要处理层级语义的场景
开放词汇语义分割需将图像级视觉-语言模型（如CLIP）适配为密集像素级预测，但嵌入空间中层级结构与语义对齐存在不匹配。尽管近期工作利用双曲几何建模层级关系，却仅对齐跨层级嵌入，忽视同层级内语义错位问题。本文提出HyRo，一种基于庞加莱球模型的双曲微调框架，解耦层次与语义对齐：通过调整双曲半径对齐层级，借助保持半径不变的正交变换实现角度对齐以优化语义关系。在标准开放词汇语义分割基准（PASCAL-Context、LVIS）上的实验表明，HyRo显著优于现有方法，达到当前最优性能。
原文摘要 · Abstract (English)
Open-vocabulary semantic segmentation requires adapting image-level vision-language models such as CLIP to dense pixel-level prediction, which is challenging due to the mismatch between hierarchical structure and semantic alignment in the embedding space. While recent works leverage hyperbolic geometry to model hierarchical relationships, they align embeddings across hierarchical levels but overlook semantic misalignment among embeddings within the same level. In this work, we propose HyRo, a hyperbolic fine-tuning framework that decouples hierarchical and semantic alignment in the Poincaré ball model. HyRo aligns hierarchical levels by adjusting the hyperbolic radius and refines semantic relationships through angular alignment using an orthogonal transformation that theoretically preserves the hyperbolic radius. Experiments on standard open-vocabulary semantic segmentation benchmarks demonstrate that HyRo achieves state-of-the-art performance over prior methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。