探究大模型时间推理的关键因素：分词质量还是时间表示？
What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
- 构建多语言多时制基准测试，涵盖五种语言和三种历法
- 低资源语言中时间分词碎片化导致准确率崩溃，高资源则稳定
- 时间线性是高资源语言推理主因，分词质量主导低资源场景
我们提出MultiTempBench，一个覆盖三种任务的多语言时间推理基准，包括日期运算、时区转换与时间关系抽取，涉及英语、德语、中文、阿拉伯语和豪萨语五种语言，以及格里高利历、希吉拉历和农历三种历法。该基准包含15,000个样本，由750个精选英文问题翻译并扩展为受控日期格式变体生成。我们评估了20个大语言模型，并引入多语言日期碎片化比率（mDFR），结合人工严重度评分校准，辅以内部时间表征的几何探针分析。结果表明，时间符号的分词质量是资源依赖型瓶颈：在低资源语言和罕见历法中，年/月/日分离被破坏，准确率急剧下降；而在高资源环境下，数字级分割通常不影响表现。跨混合效应回归显示，在高资源语言中，时间线性是时间推理最强预测因子；而在低资源语言中，碎片化影响更显著。代码开源于：https://github.com/gagan3012/mtb
原文摘要 · Abstract (English)
We present MultiTempBench, a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains $15,000$ examples built by translating $750$ curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages. Code is available at: https://github.com/gagan3012/mtb
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。