arXiv:2502.17130cs.LGcs.AI2025-02ICML被引 7

提出新型双曲树嵌入方法,解决精度与显卡兼容性难题。

Low-distortion and GPU-compatible Tree Embeddings in Hyperbolic Space

  • 基于广义Delaunay构造,强化点间角度分离度
  • 畸变率显著降低,适配深度学习硬件部署
  • 浮点扩展算术替代多精度计算,支持GPU加速

树状数据(如层级结构、本体、分类体系)的嵌入是跨领域知识表示的重要问题。双曲几何天然适合树结构嵌入,相比欧氏嵌入表现更优。近期研究已将双曲嵌入用于神经网络中的层次知识融合。现有组合构造方法存在两大瓶颈:其一,需在超球面上寻找高度分离的点,但当前方法分离度不足,影响嵌入质量;本文提出高分离Delaunay树嵌入(HS-DTE),通过在广义Delaunay框架中引入角度分离机制,有效降低嵌入畸变。其二,低畸变需更高精度,现有方法依赖多精度算术,导致无法在GPU上运行。本文改用浮点扩展算术重构组合构造,在保持高精度的同时兼容加速硬件,实现高质量且实用的双曲树嵌入。

原文摘要 · Abstract (English)

Embedding tree-like data, from hierarchies to ontologies and taxonomies, forms a well-studied problem for representing knowledge across many domains. Hyperbolic geometry provides a natural solution for embedding trees, with vastly superior performance over Euclidean embeddings. Recent literature has shown that hyperbolic tree embeddings can even be placed on top of neural networks for hierarchical knowledge integration in deep learning settings. For all applications, a faithful embedding of trees is needed, with combinatorial constructions emerging as the most effective direction. This paper identifies and solves two key limitations of existing works. First, the combinatorial construction hinges on finding highly separated points on a hypersphere, a notoriously difficult problem. Current approaches achieve poor separation, degrading the quality of the corresponding hyperbolic embedding. We propose highly separated Delaunay tree embeddings (HS-DTE), which integrates angular separation in a generalized formulation of Delaunay embeddings, leading to lower embedding distortion. Second, low-distortion requires additional precision. The current approach for increasing precision is to use multiple precision arithmetic, which renders the embeddings useless on GPUs in deep learning settings. We reformulate the combinatorial construction using floating point expansion arithmetic, leading to superior embedding quality while retaining utility on accelerated hardware.

双曲嵌入树结构GPU兼容精度优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。