构建中文朝代时间推理基准，评估大模型跨时代关系理解能力
Benchmarking Temporal Reasoning and Alignment Across Chinese Dynasties
- 基于中国朝代历史构建跨实体时间关系推理任务
- 大模型在跨朝代对齐与文化语境推理上表现有限
- 适合研究历史认知、时序理解的AI方向学者
时间推理是人类认知的基础，对现实应用至关重要。尽管大语言模型在时间推理方面展现出潜力，但现有基准多依赖规则构建，缺乏上下文深度且时间实体范围有限。为此，我们提出中文时间推理基准CTM，用于评估大模型在广阔中国朝代时间序列中的推理能力。CTM强调跨实体关系、成对时间对齐以及情境化和文化根基的推理，提供全面评估。大量实验揭示了该基准带来的挑战，并指明改进方向。
原文摘要 · Abstract (English)
Temporal reasoning is fundamental to human cognition and is crucial for various real-world applications. While recent advances in Large Language Models have demonstrated promising capabilities in temporal reasoning, existing benchmarks primarily rely on rule-based construction, lack contextual depth, and involve a limited range of temporal entities. To address these limitations, we introduce Chinese Time Reasoning (CTM), a benchmark designed to evaluate LLMs on temporal reasoning within the extensive scope of Chinese dynastic chronology. CTM emphasizes cross-entity relationships, pairwise temporal alignment, and contextualized and culturally-grounded reasoning, providing a comprehensive evaluation. Extensive experimental results reveal the challenges posed by CTM and highlight potential avenues for improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。