arXiv:2508.07185cs.CLcs.AI2025-08被引 2

让大模型实时更新知识,快且准。

DySK-Attn: A Framework for Efficient, Real-Time Knowledge Updating in Large Language Models via Dynamic Sparse Knowledge Attention

  • 用动态知识图谱+稀疏注意力,快速定位相关知识
  • 在时效问答任务中准确率超基线,计算开销更低
  • 适合需要实时知识更新的智能系统开发者

大语言模型的知识静态且易过时。重训练成本过高,现有编辑方法速度慢且可能引入副作用。为此,我们提出DySK-Attn框架,使大模型能高效融合来自动态外部知识源的实时知识。该框架将大模型与可即时更新的知识图谱(KG)结合,核心是稀疏知识注意力机制,实现从粗到细的知识搜索,仅聚焦于知识库中少量高相关事实。该机制避免对整个知识库进行密集注意力计算,降低计算开销并减少无关信息干扰。在时间敏感的问答任务上,大量实验表明,DySK-Attn显著优于强基线,包括标准检索增强生成(RAG)和模型编辑技术,在新知识准确性与计算效率方面均表现更优。本框架为构建持续更新的大模型提供了可扩展、高效的解决方案。

原文摘要 · Abstract (English)

Large Language Models (LLMs) suffer from a critical limitation: their knowledge is static and quickly becomes outdated. Retraining these massive models is computationally prohibitive, while existing knowledge editing techniques can be slow and may introduce unforeseen side effects. To address this, we propose DySK-Attn, a novel framework that enables LLMs to efficiently integrate real-time knowledge from a dynamic external source. Our approach synergizes an LLM with a dynamic Knowledge Graph (KG) that can be updated instantaneously. The core of our framework is a sparse knowledge attention mechanism, which allows the LLM to perform a coarse-to-fine grained search, efficiently identifying and focusing on a small, highly relevant subset of facts from the vast KG. This mechanism avoids the high computational cost of dense attention over the entire knowledge base and mitigates noise from irrelevant information. We demonstrate through extensive experiments on time-sensitive question-answering tasks that DySK-Attn significantly outperforms strong baselines, including standard Retrieval-Augmented Generation (RAG) and model editing techniques, in both factual accuracy for updated knowledge and computational efficiency. Our framework offers a scalable and effective solution for building LLMs that can stay current with the ever-changing world.

大模型知识更新稀疏注意力动态知识图谱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。