让知识检索有顺序,提升语言模型推理能力
Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Models
- 用带顺序的超图表示知识,把检索变成序列推断
- 在气象和港口任务中,准确率显著高于传统方法
- 适合需要逻辑顺序的任务,如复杂决策解释
检索增强生成(RAG)通过引入外部知识提升大语言模型性能。然而,现有基于图或超图的方法将检索证据视为无序集合,隐含假设排列不变性,这与许多真实推理任务不符——结果不仅取决于哪些交互发生,还取决于其发生顺序。我们提出有序感知的知识超图RAG(OKH-RAG),将顺序作为结构核心属性。OKH-RAG在超图中引入先后关系,将检索重构为对超边序列的推理。它不选择独立事实,而是恢复反映底层推理过程的连贯交互轨迹。一个学习到的转移模型直接从数据中推断顺序,无需显式时间标注。我们在依赖顺序的问题回答与解释任务上评估,包括热带气旋和港口运营场景。OKH-RAG持续优于排列不变基线,消融实验表明性能提升正是源于对交互顺序的建模。结果揭示了集合型检索的关键局限:有效推理不仅需要获取相关证据,还需将其组织成结构化序列。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) enhances large language models by grounding outputs in retrieved knowledge. However, existing RAG methods including graph- and hypergraph-based approaches treat retrieved evidence as an unordered set, implicitly assuming permutation invariance. This assumption is misaligned with many real-world reasoning tasks, where outcomes depend not only on which interactions occur, but also on the order in which they unfold. We propose Order-Aware Knowledge Hypergraph RAG (OKH-RAG), which treats order as a first-class structural property. OKH-RAG represents knowledge as higher-order interactions within a hypergraph augmented with precedence structure, and reformulates retrieval as sequence inference over hyperedges. Instead of selecting independent facts, it recovers coherent interaction trajectories that reflect underlying reasoning processes. A learned transition model infers precedence directly from data without requiring explicit temporal supervision. We evaluate OKH-RAG on order-sensitive question answering and explanation tasks, including tropical cyclone and port operation scenarios. OKH-RAG consistently outperforms permutation-invariant baselines, and ablations show that these gains arise specifically from modeling interaction order. These results highlight a key limitation of set-based retrieval: effective reasoning requires not only retrieving relevant evidence, but organizing it into structured sequences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。