用双曲空间建模类别层次,提升未知物体识别能力。
Hyp2Former: Hierarchy-Aware Hyperbolic Embeddings for Open-Set Panoptic Segmentation

- 在双曲空间中学习已知类别的层次语义关系
- 未知物体与高层概念距离更近,提升检测可靠性
- 无需训练时显式建模未知类,适合真实场景应用
识别未知物体对自动驾驶和机器人等安全关键应用至关重要。开放集全景分割(OPS)旨在分割已知的物体内部与背景,并将有效未知物体作为独立实例识别。现有方法多将已知类别视为平铺标签集,忽略其语义层次结构,而该结构对区分未知物体与分布内类别具有重要先验作用。本文提出Hyp2Former,一种端到端的OPS框架,无需在训练时显式建模未知类,而是持续在双曲空间中学习类别间的层次语义相似性。通过显式编码已知类别间的层次关系,模型构建出具备多级语义抽象能力的嵌入空间。结果表明,即使细粒度类别未在训练中出现,未知物体仍能保持与高层概念(如“动物”或“物体”)的紧密接近,从而被可靠检测。在MS COCO、Cityscapes和Lost&Found等多个公开数据集上的实验显示,Hyp2Former在未知物体发现与分布内鲁棒性之间达到最佳平衡,优于现有方法。
原文摘要 · Abstract (English)
Recognizing unknown objects is crucial for safety-critical applications such as autonomous driving and robotics. Open-Set Panoptic Segmentation (OPS) aims to segment known thing and stuff classes while identifying valid unknown objects as separate instances. Prior OPS approaches largely treat known categories as a flat label set, ignoring the semantic hierarchy that provides valuable structural priors for distinguishing unknown objects from in-distribution classes. In this work, we propose Hyp2Former, an end-to-end framework for OPS that does not require explicit modeling of unknowns during training, and instead learns hierarchical semantic similarities continuously in hyperbolic space. By explicitly encoding hierarchical relationships among known categories, the model learns a structured embedding space that captures multiple levels of semantic abstraction. As a result, unknown objects that cannot be confidently classified as known categories still remain in close proximity to higher-level concepts (e.g., an unknown animal remains closer to "animal" or "object" than to unrelated concepts such as "electronics" or "stuff") and can therefore be reliably detected, even if their fine-grained category was not represented during training. Empirical evaluations across multiple public datasets such as MS COCO, Cityscapes, and Lost&Found demonstrate that Hyp2Former outperforms existing methods on OPS, achieving the best balance between unknown object discovery and in-distribution robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。