arXiv:2412.01023cs.LGcs.CV2024-12NeurIPS被引 22

用双曲空间建模标签层级,提升低维下的表示准确性与泛化能力。

Learning Structured Representations with Hyperbolic Embeddings

  • 引入双曲树结构损失和中心化损失,显式编码标签层级信息。
  • 在多个大规模视觉数据集上显著降低表示扭曲,尤其在低维时提升性能。
  • 适用于需要结构感知表示的场景,如小样本学习与分布外检测。

现实世界数据集通常具有天然的类别层次结构或标签结构,这些结构可低成本构建。然而现有表示学习方法普遍忽略此结构,将标签视为置换不变。近期工作 [Zeng et al., 2022] 提出显式利用结构信息,但使用欧氏距离可能扭曲语义上下文 [Chen et al., 2013]。本文受双曲空间对层次关系建模优势的启发,提出 HypStructure:一种双曲结构正则化方法,可将标签层级准确嵌入学习到的表示中。HypStructure 是一个简单而有效的正则项,包含基于双曲树的表示损失与中心化损失,可与任意标准任务损失结合,学习结构感知特征。在多个大规模视觉基准上的实验表明,该方法能有效减少表示扭曲,并在低维场景下提升泛化性能。通过特征值分析,我们揭示了表示几何与分布外检测性能提升之间的关联。代码已公开于 url{https://github.com/uiuctml/HypStructure}。

原文摘要 · Abstract (English)

Most real-world datasets consist of a natural hierarchy between classes or an inherent label structure that is either already available or can be constructed cheaply. However, most existing representation learning methods ignore this hierarchy, treating labels as permutation invariant. Recent work [Zeng et al., 2022] proposes using this structured information explicitly, but the use of Euclidean distance may distort the underlying semantic context [Chen et al., 2013]. In this work, motivated by the advantage of hyperbolic spaces in modeling hierarchical relationships, we propose a novel approach HypStructure: a Hyperbolic Structured regularization approach to accurately embed the label hierarchy into the learned representations. HypStructure is a simple-yet-effective regularizer that consists of a hyperbolic tree-based representation loss along with a centering loss, and can be combined with any standard task loss to learn hierarchy-informed features. Extensive experiments on several large-scale vision benchmarks demonstrate the efficacy of HypStructure in reducing distortion and boosting generalization performance especially under low dimensional scenarios. For a better understanding of structured representation, we perform eigenvalue analysis that links the representation geometry to improved Out-of-Distribution (OOD) detection performance seen empirically. The code is available at \url{https://github.com/uiuctml/HypStructure}.

双曲嵌入层次结构表示学习低维表示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。