让大模型学会处理随时间变化的矛盾信息,提升动态知识推理能力。
Question Answering under Temporal Conflict: Evaluating and Organizing Evolving Knowledge with LLMs
- 构建时序记忆框架,从文档中逐步提取并组织时间相关的知识
- 在两个新基准上显著优于传统方法,复杂问题准确率提升12%-18%
- 无需重训练,适合需要实时更新知识的场景如新闻、体育赛事
大型语言模型凭借其参数化记忆在问答与推理任务中表现优异,但其知识受限于预训练数据范围,而现实世界信息持续演化。常规更新方式需代价高昂且脆弱的重训练,或难以规模化应用的上下文学习(ICL)。为此,我们研究了模型在时序文本语料(如逐年更新的运动员生涯记录)下的表现,提出两个新基准:Temporal Wiki(基于历史维基快照的事实演变)和Unified Clark(聚合带时间戳的新闻文章模拟信息积累)。分析发现,模型常无法调和冲突或过时信息,易受多版本事实干扰。为此,我们设计一种轻量级、代理式框架,无需重训练即可从源文档增量构建结构化外部记忆。该机制使模型能在推理时检索并处理时间过滤后的相关知识。实验表明,该方法在两个基准上均优于ICL与RAG基线,尤其在涉及复杂推理或冲突事实整合的问题上表现更优。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit remarkable capabilities in question answering and reasoning thanks to their extensive parametric memory. However, their knowledge is inherently limited by the scope of their pre-training data, while real-world information evolves continuously. Updating this knowledge typically requires costly and brittle re-training, or in-context learning (ICL), which becomes impractical at scale given the volume and volatility of modern information. Motivated by these limitations, we investigate how LLMs perform when exposed to temporal text corpora, or documents that reflect evolving knowledge over time, such as sports biographies where facts like a player's "current team" change year by year. To this end, we introduce two new benchmarks: Temporal Wiki, which captures factual drift across historical Wikipedia snapshots, and Unified Clark, which aggregates timestamped news articles to simulate real-world information accumulation. Our analysis reveals that LLMs often struggle to reconcile conflicting or outdated facts and can be misled when multiple versions of a fact appear in context. To address these issues, we propose a lightweight, agentic framework that incrementally builds a structured, external memory from source documents without requiring re-training. This knowledge organization strategy enables models to retrieve and reason over temporally filtered, relevant information at inference time. Empirically, our method outperforms ICL and RAG baselines across both benchmarks, especially on questions requiring more complex reasoning or integration of conflicting facts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。