arXiv:2511.14214cs.AIcs.LG2025-11

测试大模型是否真懂时间顺序,发现推理能力决定能否准确排序。

Do Large Language Models (LLMs) Understand Chronology?

  • 设计三类时序任务:排序、条件排序、错位检测,验证模型时间理解力。
  • 长序列排序准确率骤降,但局部顺序仍保持,全局时间线难维持。
  • 增加推理预算后,GPT-5表现近乎完美,适合金融等实时场景应用。

大语言模型在金融与经济领域应用日益广泛,其提示方法隐含假设模型具备时间理解能力。本文通过一系列复杂度递增的时序排序任务,测试模型对已有预训练知识的时间把握能力。任务涵盖(1)时序排序,(2)条件排序(先筛选再排序),(3)错位事件识别。评估 GPT-4.1、Claude-3.7 Sonnet 及 GPT-5 在不同推理强度下的表现。结果表明,随着序列长度增加,精确匹配率急剧下降,但排名相关性保持较高,说明模型虽能维护局部顺序,却难以构建统一全局时间线。条件排序失败主要源于筛选环节而非排序本身;而开启扩展推理(ET)后,GPT-5 和 Claude-3.7 Sonnet 显著优于普通模式。错位检测为最易任务,但重叠时间线或实体仍导致性能下降。核心贡献在于揭示:合理分配推理资源可显著提升时序排序效果——GPT-5 在中高推理强度下实现全长度无误排序与完美条件排序(自筛选与给定子集均成立),而低强度则随序列增长迅速退化,再现早期模型局限。研究厘清了当前模型在时序任务中的边界,为金融等领域实时应用提供关键参考。所有代码与评估模板已开源,确保可复现。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used in finance and economics, where prompt-based attempts against look-ahead bias implicitly assume that models understand chronology. We test this fundamental question with a series of chronological ordering tasks with increasing complexities over facts the model already knows from pre-training. Our tasks cover (1) chronological ordering, (2) conditional sorting (filter, then order), and (3) anachronism detection. We evaluate GPT-4.1, Claude-3.7 Sonnet, with and without Extended Thinking (ET), and GPT-5 across multiple reasoning-effort settings. Across models, Exact match rate drops sharply as sequences lengthen even while rank correlations stay high as LLMs largely preserve local order but struggle to maintain a single globally consistent timeline. In conditional sorting, most failures stem from the filtering step rather than the ordering step, but GPT-5 and Claude-3.7 Sonnet with Extended Thinking outshine normal models significantly. Lastly, anachronism detection is found to be the easiest task for the LLMs but performance still declines with increasingly overlapping timelines or entities. Overall, our main contribution is showing that allocating explicit reasoning budget helps with chronological ordering with GPT-5 at medium/high reasoning effort achieving flawless ordering at all lengths and perfect conditional sorting (both self-filtered and given-subset), whereas low/minimal effort degrades with longer lists, mirroring earlier models. Our findings delineate limits of current LLMs on chronological tasks, providing insights into task complexity, and demonstrate scenarios in which reasoning helps. These patterns are important for the real-time application of LLMs in finance. We release all code and evaluation templates to support full reproducibility.

时序理解大模型评测推理能力金融AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。