arXiv:2608.15507cs.CLcs.LG2026-08中稿 · COLM

语言模型对当前年份的编码不一致,导致更新困难。

Do Language Models Consistently Encode the Current Year?

论文配图:Do Language Models Consistently Encode the Current Year?
图 1 · 摘自论文原文
  • 用动词时态和直接提问两种方式探测模型对当前年份的理解。
  • 基础模型在关联任务中误差仅10个月,但声明任务无法稳定更新。
  • 预训练时的语义结构难以修改,提示、微调和权重编辑均无效。

一致的时间概念对时间推理至关重要,但语言模型如何表示当前时间尚不明确。本文设计两项概念不同的探测任务:基于动词时态推断当前年份的关联任务,以及直接询问当前年份的声明任务。两者均能将语言模型的当前年份估计在指令微调数据截止时间的一年内。对于基础模型,关联任务的预测可作为预训练数据截止时间的强代理,13个模型平均误差仅10个月。然而内部机制存在差异:关联任务依赖类似事实回忆的机制,而声明任务缺乏稳定的因果路径。这导致更新当前年份面临挑战——提示、SFT或权重编辑均无法同时改变两个任务的结果。提示成功更新声明年份(351个目标年份中94.6%成功),但关联年份几乎不变(1.7%成功)。年份调整的SFT也仅在8个模型中有一个匹配目标年份。权重编辑虽对单个任务有效,但无法跨任务泛化。总体表明,当前年份在语言模型中未被一致编码:关联概念深植于预训练语言结构,使用不同因果机制,对与声明任务相同的修改具有抗性。

原文摘要 · Abstract (English)

A consistent concept of the current time is important for temporal reasoning, yet how language models represent the current time is not well understood. We contribute two tasks that probe the current year in conceptually distinct ways: an associative task, which infers the current year from verb tense, and a declarative task, which directly queries for the current year. Both tasks estimate current years within one year of the post-training data cutoff of instruction-tuned language models. For base models, predictions on the associative task serve as a strong proxy for the pre-training data cutoff, with an average error of only 10 months across 13 models. However, their internal mechanisms diverge: the associative task uses mechanisms similar to factual recall, while the declarative task lacks consistent causal pathways. This divergence poses a challenge for updating the current year in language models. None of prompting, SFT, or weight editing succeed in shifting the associative and declarative years simultaneously. Prompting updates the declarative year (94.6% success across 351 target years) but leaves the associative year nearly unchanged (1.7% success). Year-shifted SFT also fails to shift the associative year, matching the target year in only one of eight models. Weight editing, while effective for both tasks individually, does not generalize across both. Overall, our results show that the current year is not consistently encoded in language models: The associative notion, deeply ingrained in linguistic structures learned in pre-training, uses different causal mechanisms and resists the same modifications that easily shift the declarative notion learned in post-training.

语言模型时间编码推理模型更新

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。