测试大模型在多轮对话中保持时间一致性能力，发现其常错误默认当前时间。
Evaluating Temporal Consistency in Multi-Turn Language Models
- 构建百万级可控对话数据集，专测时间范围延续与切换能力。
- 主流模型在长对话中频繁误用现时假设，准确率下降超40%。
- 适合研究对话系统、时间推理或评估模型长期一致性的人参考。
语言模型越来越多地应用于交互场景，用户需在时间维度上持续推理事实。这要求模型能维持并更新对话早期建立的隐含时间假设。我们通过时间范围稳定性（temporal scope stability）来研究这一挑战：即在多轮对话中保留、覆盖或转移时间限定的事实上下文的能力。为此，我们提出了 ChronoScope——一个大规模诊断基准，包含超过一百万条基于 Wikidata 生成的确定性问题链，用于隔离时间范围行为。该基准评估模型在省略显式时间参考的后续问题中，能否正确保持推断出的时间范围，涵盖隐式延续、显式范围切换、跨实体传递及更长的时间轨迹。对先进语言模型的广泛评估显示，时间范围稳定性在受控多轮设置中经常被破坏，模型常错误地趋向于当前时间假设，尽管其底层知识是正确的。这种错误随对话长度增加而加剧，并在提供理想上下文条件下依然存在，揭示了单轮事实准确性与序列交互中连贯时间推理之间的差距。数据集与评估工具已公开于 https://github.com/yashkumaratri/ChronoScope。
原文摘要 · Abstract (English)
Language models are increasingly deployed in interactive settings where users reason about facts over time rather than in isolation. In such scenarios, correct behavior requires models to maintain and update implicit temporal assumptions established earlier in a conversation. We study this challenge through the lens of temporal scope stability: the ability to preserve, override, or transfer time-scoped factual context across dialogue turns. We introduce ChronoScope, a large-scale diagnostic benchmark designed to isolate temporal scope behavior in controlled multi-turn interactions, comprising over one million deterministically generated question chains grounded in Wikidata. ChronoScope evaluates whether models can correctly retain inferred temporal scope when follow-up questions omit explicit time references, spanning implicit carryover, explicit scope switching, cross-entity transfer, and longer temporal trajectories. Through extensive evaluation of state-of-the-art language models, we find that temporal scope stability is frequently violated in controlled multi-turn settings, with models often drifting toward present-day assumptions despite correct underlying knowledge. These failures intensify with interaction length and persist even under oracle context conditions, revealing a gap between single-turn factual accuracy and coherent temporal reasoning under sequential interaction. We make our dataset and evaluation suite publicly available at https://github.com/yashkumaratri/ChronoScope
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。