arXiv:2602.07739cs.IRcs.AI2026-02被引 1

用双曲空间提升检索生成的准确性,降低幻觉风险。

HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generation

  • 采用双曲几何建模语言层次结构,改进密集检索
  • 在MTEB和RAGBench上分别实现29%的性能提升
  • 适合关注生成质量与模型效率的研究者

嵌入几何对检索质量具有根本影响,但当前用于检索增强生成(RAG)的密集检索器仍主要局限于欧氏空间。然而,自然语言具有从宽泛主题到具体实体的层次结构,欧氏嵌入难以保留此类结构,导致语义相距较远的文档看似相似,增加幻觉风险。为此,我们提出双曲密集检索,在洛伦兹模型的双曲空间中构建两种模型变体:完全双曲的Transformer(HyTE-FH)和将预训练欧氏嵌入投影至双曲空间的混合架构(HyTE-H)。为防止序列聚合时的表征坍塌,引入几何感知的外向爱因斯坦中点池化算子,可保证层次结构的保持。在MTEB基准上,HyTE-FH优于等效欧氏基线;在RAGBench上,HyTE-H使用更小模型即实现上下文相关性和答案相关性高达29%的提升。分析表明,双曲表示通过范数差异体现文档特异性,从一般概念到具体概念的径向长度增加超过20%,而欧氏嵌入不具备此特性,凸显几何归纳偏置在真实RAG系统中的关键作用。

原文摘要 · Abstract (English)

Embedding geometry plays a fundamental role in retrieval quality, yet dense retrievers for retrieval-augmented generation (RAG) remain largely confined to Euclidean space. However, natural language exhibits hierarchical structure from broad topics to specific entities that Euclidean embeddings fail to preserve, causing semantically distant documents to appear spuriously similar and increasing hallucination risk. To address these limitations, we introduce hyperbolic dense retrieval, developing two model variants in the Lorentz model of hyperbolic space: HyTE-FH, a fully hyperbolic transformer, and HyTE-H, a hybrid architecture projecting pre-trained Euclidean embeddings into hyperbolic space. To prevent representational collapse during sequence aggregation, we introduce the Outward Einstein Midpoint, a geometry-aware pooling operator that provably preserves hierarchical structure. On MTEB, HyTE-FH outperforms equivalent Euclidean baselines, while on RAGBench, HyTE-H achieves up to 29% gains over Euclidean baselines in context relevance and answer relevance using substantially smaller models than current state-of-the-art retrievers. Our analysis also reveals that hyperbolic representations encode document specificity through norm-based separation, with over 20% radial increase from general to specific concepts, a property absent in Euclidean embeddings, underscoring the critical role of geometric inductive bias in faithful RAG systems.

双曲嵌入检索增强生成质量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。