构建动态时间序列分析基准，测试模型随时间演进的推理能力
TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments
- 设计实时更新的60个真实场景基准，覆盖6大领域
- 1485组问答对验证模型在时间流中的状态识别与展望推理
- 适合评估大模型在持续变化数据中的适应性与时效性
高风险领域的时间序列分析依赖定期数据发布，新数据可能改变证据基础并影响后续结论的有效性。现有时间序列问答基准多基于固定快照，未能评估时间有效性与截止点感知的证据使用。我们提出TimeSage-EV，一个面向动态环境的智能体时间序列分析实时基准。它跟踪60个真实机构场景，涵盖6个领域，包含从2023年2月至2026年5月的1,485组场景-周期问答对，覆盖月度、周度、日度及非规则发布频率。每个周期中，大语言模型（LLM）智能体接收时间序列数据和源报告，而被隐藏的目标发布数据作为真实标签。TimeSage-EV评估状态识别、数据摘要与未来推断能力。对前沿LLM智能体及TimeSage-1.0——一种具有可复用分析技能库的自演化智能体的实验显示，不同模型层级间存在显著性能差距，且普遍存在时间有效性缺失、外部上下文利用不足与适应性差的问题。我们公开TimeSage-EV，提供每月更新、代码、排行榜与错误模式分析。
原文摘要 · Abstract (English)
Time series analysis in high-stakes domains relies on recurring data releases, where new observations can alter the evidence base and the validity of later conclusions. Existing time series QA benchmarks mostly rely on fixed snapshots, leaving temporal validity and cutoff-aware evidence use unevaluated. We introduce TimeSage-EV, a live benchmark for agentic time series analysis in evolving environments. It tracks 60 real institutional scenarios across 6 domains, comprising 1,485 scenario-period QA pairs from Feb 2023 to May 2026 and spanning monthly, weekly, daily, and irregular release cadences. At each period, large language model (LLM) agents receive time series data and source reports, while the withheld target release provides ground truth. TimeSage-EV evaluates state identification, data summarization, and outlook reasoning. Experiments with frontier LLM agents and TimeSage-1.0, a novel self-evolving agent with a reusable analytical skill library, reveal significant performance gaps across model tiers and recurring failures in temporal validity, exogenous context use, and adaptation. We release TimeSage-EV as a research resource with monthly updates, code, a leaderboard, and failure-mode analyses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。