评测大模型在中文法律中的时间推理能力，发现现有模型仍有明显短板。
LexKairos: Benchmarking Legal Temporal Capabilities in LLMs
- 构建涵盖法规、案件、推理的三维度中文法律时间基准测试
- 最佳模型仅在简单时间任务表现尚可，复杂时限推理仍差
- 适合法律AI研究者与需处理时效性法律问题的开发者
大型语言模型（LLMs）在众多法律任务中展现出强大性能。在法律实践中，时间是决定法规有效性、案件进展及程序期限执行的关键概念。然而，现有法律AI基准对法律时间能力的考察仍显不足。为此，我们提出LexKairos，一个针对中文法律语境下大模型时间能力的综合性评估基准，涵盖法规时间知识、案件时间建模与法规-案件时间推理三个维度。该基准包含九个来自真实中国司法案例与法规的子任务。我们对八种LLMs在多种推理模式（包括原始、思维链CoT及思考模式）下进行了系统评估。结果表明，Gemini-3-Flash整体表现最优，但即便如此，在需要精确时间敏感法规元数据召回或复杂时限推理的任务上仍存在显著局限，表明当前大模型在法律时间知识与推理方面仍是开放挑战。数据与代码已公开于https://github.com/thunlp/LexKairos。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that governs the validity of statutes, the progression of legal cases, and the enforcement of procedural deadlines. However, legal temporal capabilities remain underexplored in existing legal AI benchmarks. To address this gap, we propose LexKairos, a comprehensive benchmark for evaluating the temporal capabilities of LLMs in the Chinese legal context across three dimensions: statutory temporal knowledge, case temporal modeling, and statute-case temporal reasoning. LexKairos comprises nine sub-tasks drawn from real-world Chinese judicial cases and statutes. We conduct systematic evaluations of eight LLMs under multiple inference settings, including vanilla, Chain-of-Thought (CoT), and thinking modes. Our results show that Gemini-3-Flash achieves the strongest overall performance, yet even the best-performing model exhibits notable limitations on tasks demanding precise time-sensitive statutory metadata recall or complex reasoning in time limits, indicating that legal temporal knowledge and reasoning remain open challenges for current LLMs. Data and code are available at https://github.com/thunlp/LexKairos.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。