arXiv:2601.10863cs.LG2026-01

提出新评估指标,让预测模型更稳定准确

Beyond Accuracy: A Stability-Aware Metric for Multi-Horizon Forecasting

  • 设计兼顾准确率与时间一致性的评分机制
  • 多步预测误差降低6%,短期预测误差仅增3.9%
  • 适合需要长期稳定预测的工业场景

传统时间序列预测方法仅优化准确性,忽略了时间一致性——即随着预测起点变化,模型对同一未来事件的预测是否稳定。本文提出预测准确性和一致性得分(forecast AC score),用于衡量概率性多时域预测的质量,同时考虑多时域准确性和稳定性。该评分支持用户自定义权重,平衡准确与一致需求。作为应用,我们将该评分作为可微目标函数,用于训练季节性自回归积分模型,并在M4 Hourly基准数据集上进行评估。结果表明,相较于传统的最大似然估计,模型在稳定性上显著提升:对相同时间戳的外样本预测方差降低了15.8%;在准确性方面,中长期预测有明显改善,从第3个时域起误差下降,第9-12时域达到约6%的最低均方误差降幅;而一步预测仅增加3.9%的均方误差。这表明该指标能有效训练出更稳定、更准确的多步预测模型,且对一步预测影响较小。

原文摘要 · Abstract (English)

Traditional time series forecasting methods optimize for accuracy alone. This objective neglects temporal consistency, in other words, how consistently a model predicts the same future event as the forecast origin changes. We introduce the forecast accuracy and coherence score (forecast AC score for short) for measuring the quality of probabilistic multi-horizon forecasts in a way that accounts for both multi-horizon accuracy and stability. Our score additionally allows user-specified weights to balance accuracy and consistency requirements. As an example application, we implement the score as a differentiable objective function for training seasonal auto-regressive integrated models and evaluate it on the M4 Hourly benchmark dataset. Results demonstrate consistent improvements over traditional maximum likelihood estimation. Regarding stability, the AC-optimized model generated out-of-sample forecasts with 15.8\% reduced variance over forecasts targeting the same timestamp. In terms of accuracy, the AC-optimized model achieved considerable improvements for medium-to-long-horizon forecasts. While one-step-ahead forecasts exhibited a 3.9\% increase in MSE, forecasts from horizon three onward experienced improved accuracy, with a peak improvement of approximately 6\% in MSE at horizons 9-12. These results indicate that our metric successfully trains models to produce more stable and accurate multi-step forecasts in exchange for a relatively small degradation in one-step-ahead performance.

时间序列预测评估稳定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。