arXiv:2505.16088cs.CLcs.AI2025-05EMNLP被引 8

日期分段问题影响时间推理,模型需自动生成完整日期结构

Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning

  • 提出日期碎片率衡量分词器对日期结构的保留程度
  • 发现过度分段导致罕见日期推理准确率下降最高10分
  • 揭示大模型能自动拼合日期片段,路径不同于人类

现代BPE分词器常将日历日期切分为无意义片段,如20250312被拆为202、503、12,增加分词数量并破坏时间结构。本文引入可解释的日期碎片率指标,评估分词器对多数字日期成分的保留能力;发布DateAugBench,包含6500个样本,覆盖上下文日期解析、格式不变谜题和跨历史、当代、未来的时间算术任务;通过层间探测与因果注意力分析,发现大语言模型会自发形成日期抽象机制,拼合年、月、日片段进行推理。实验显示,过度碎片化导致罕见日期(如历史或未来日期)准确率下降最高达10分。且模型越大,该修复机制越快显现。此外,模型组装日期的路径通常与人类不同(如年→月→日)。数据集与代码已公开。

原文摘要 · Abstract (English)

Modern BPE tokenizers often split calendar dates into meaningless fragments, e.g., 20250312 $\rightarrow$ 202, 503, 12, inflating token counts and obscuring the inherent structure needed for robust temporal reasoning. In this work, we (1) introduce a simple yet interpretable metric, termed date fragmentation ratio, that measures how faithfully a tokenizer preserves multi-digit date components; (2) release DateAugBench, a suite of 6500 examples spanning three temporal reasoning tasks: context-based date resolution, format-invariance puzzles, and date arithmetic across historical, contemporary, and future time periods; and (3) through layer-wise probing and causal attention-hop analyses, uncover an emergent date-abstraction mechanism whereby large language models stitch together the fragments of month, day, and year components for temporal reasoning. Our experiments show that excessive fragmentation correlates with accuracy drops of up to 10 points on uncommon dates like historical and futuristic dates. Further, we find that the larger the model, the faster the emergent date abstraction that heals date fragments is accomplished. Lastly, we observe a reasoning path that LLMs follow to assemble date fragments, typically differing from human interpretation (year $\rightarrow$ month $\rightarrow$ day). Our datasets and code are made publicly available \href{https://github.com/gagan3012/date-fragments}{here}.

时间推理分词器大模型机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。