arXiv:2507.08704cs.CLcs.AI2025-07被引 1

让大模型实时获取最新知识,不改参数也能动态更新。

Knowledge Fusion via Bidirectional Information Aggregation

  • 通过双向路径在推理时融合外部知识图谱,不修改模型参数。
  • 在四个基准上实现高效知识融合,保持模型原有能力。
  • 适合需要实时知识更新的网页应用,如智能问答、推荐系统。

知识图谱(KGs)是语义网的核心,提供对现实世界实体与关系的实时表示。然而大语言模型(LLMs)在预训练后保持静态,其内部知识易过时,限制了其在时效性任务中的应用。现有方法通常依赖参数侵入式微调,易导致灾难性遗忘并损害模型泛化能力。同时,其静态集成框架难以跟上现实世界知识图谱的持续演化,制约了在动态网络环境中的部署。为此,我们提出KGA(知识图谱引导注意力),一种仅在推理阶段动态融合外部知识图谱的新框架,无需任何参数修改。受神经科学启发,我们重构自注意力模块,引入两个协同路径:自底向上的知识融合路径,通过输入驱动的外部知识整合到输入表示中,类比人脑的刺激驱动注意机制;自顶向下的注意力引导路径,通过目标导向的验证过程评估每条三元组的上下文相关性,抑制无关信号,增强知识相关模式。二者协同实现实时知识融合。在四个基准上的大量实验验证了KGA的强融合性能与高效率。

原文摘要 · Abstract (English)

Knowledge graphs (KGs) are the cornerstone of the semantic web, offering up-to-date representations of real-world entities and relations. Yet large language models (LLMs) remain largely static after pre-training, causing their internal knowledge to become outdated and limiting their utility in time-sensitive web applications. To bridge this gap between dynamic knowledge and static models, a prevalent approach is to enhance LLMs with KGs. However, prevailing methods typically rely on parameter-invasive fine-tuning, which risks catastrophic forgetting and often degrades LLMs' general capabilities. Moreover, their static integration frameworks cannot keep pace with the continuous evolution of real-world KGs, hindering their deployment in dynamic web environments. To bridge this gap, we introduce KGA (\textit{\underline{K}nowledge \underline{G}raph-guided \underline{A}ttention}), a novel framework that dynamically integrates external KGs into LLMs exclusively at inference-time without any parameter modification. Inspired by research on neuroscience, we rewire the self-attention module by innovatively introducing two synergistic pathways: a \textit{bottom-up knowledge fusion} pathway and a \textit{top-down attention guidance} pathway. The \textit{bottom-up pathway} dynamically integrates external knowledge into input representations via input-driven KG fusion, which is akin to the \textit{stimulus-driven attention process} in the human brain. Complementarily, the \textit{top-down pathway} aims to assess the contextual relevance of each triple through a \textit{goal-directed verification process}, thereby suppressing task-irrelevant signals and amplifying knowledge-relevant patterns. By synergistically combining these two pathways, our method supports real-time knowledge fusion. Extensive experiments on four benchmarks verify KGA's strong fusion performance and efficiency.

知识图谱大模型动态融合注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。