首个评估预测系统推理能力的基准，让模型不仅能算对，还能说清理由。
TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems
- 构建多智能体框架，通过迭代验证生成可量化的推理过程。
- 用生成的推理提示，模型预测准确率从40.2%提升至56.6%。
- 适合关注可解释性、因果推理的预测系统研究者使用。
我们提出TFRBench，首个用于评估预测系统推理能力的基准。传统时间序列预测仅关注数值准确性，将基础模型视为“黑箱”。与现有基准不同，TFRBench提供评估预测系统推理过程的协议，聚焦其对跨通道依赖、趋势和外部事件的分析。为此，我们设计了基于多智能体的系统性框架，通过迭代验证循环生成数值支撑的推理轨迹。在五个领域共十个数据集上的评估表明，此类推理具有因果有效性，可用于评估，并显著提升大模型预测精度（如平均准确率由40.2%提升至56.6%）。反观，现成大模型在推理（判别得分低）和数值预测上均表现不佳，难以捕捉领域特异性动态。TFRBench因此确立了时间序列预测中可解释、基于推理的新评估标准。基准地址：https://tfrbench.github.io
原文摘要 · Abstract (English)
We introduce TFRBench, the first benchmark designed to evaluate the reasoning capabilities of forecasting systems. Traditionally, time-series forecasting has been evaluated solely on numerical accuracy, treating foundation models as ``black boxes.'' Unlike existing benchmarks, TFRBench provides a protocol for evaluating the reasoning generated by forecasting systems--specifically their analysis of cross-channel dependencies, trends, and external events. To enable this, we propose a systematic multi-agent framework that utilizes an iterative verification loop to synthesize numerically grounded reasoning traces. Spanning ten datasets across five domains, our evaluation confirms that this reasoning is causally effective; useful for evaluation; and prompting LLMs with our generated traces significantly improves forecasting accuracy compared to direct numerical prediction (e.g., avg. $\sim40.2\%\to56.6\%)$, validating the quality of our reasoning. Conversely, benchmarking experiments reveal that off-the-shelf LLMs consistently struggle with both reasoning (lower LLM-as-a-Judge scores) and numerical forecasting, frequently failing to capture domain-specific dynamics. TFRBench thus establishes a new standard for interpretable, reasoning-based evaluation in time-series forecasting. Our benchmark is available at: https://tfrbench.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。