arXiv:2608.17293cs.LGcs.AI2026-08

提出新评估指标CSE,解决不规则时间序列预测中MSE的偏差问题。

Beyond MSE: Rethinking the Evaluation Metric and Benchmarking for Irregular Time Series Forecasting

论文配图:Beyond MSE: Rethinking the Evaluation Metric and Benchmarking for Irregular Time Series Forecasting
图 1 · 摘自论文原文
  • 引入重要性加权的连续时间平方误差(CSE)
  • CSE比MSE更准确反映模型在连续时间上的真实性能
  • 涵盖8个真实数据集的系统性基准测试

不规则时间序列预测研究长期聚焦于模型设计,而评估指标缺乏深入探讨。现有基准普遍采用均方误差(MSE)作为评价标准。我们发现,在不规则预测中,MSE不仅受模型预测影响,还受样本特定的时间戳采样分布影响,导致对模型连续时间预测性能的评估存在偏差。为此,我们提出连续时间平方误差(CSE),通过重要性加权消除时间戳采样分布的影响。理论证明,CSE在渐近估计误差上不劣于MSE,且更接近连续时间风险。我们构建了包含合成、半合成及八个真实数据集的系统性基准,验证了CSE的有效性,并系统评估模型的连续时间预测表现。实验表明,CSE能更准确恢复连续时间风险,仅依赖MSE可能无法充分反映真实场景下的模型性能。代码已开源:https://github.com/hnu-vis/ITS-Bench。

原文摘要 · Abstract (English)

Existing research on irregular time-series forecasting has primarily focused on model design, while evaluation metrics remain insufficiently studied. Existing benchmarks typically use mean squared error (MSE) as the evaluation metric. We show that, in irregular forecasting, MSE is determined not only by the model prediction but also by the sample-specific timestamp sampling distributions, leading to a biased assessment of the models' continuous-time predictive performance. To address this issue, we propose the Continuous-time Squared Error (CSE), which employs importance weighting to eliminate the influence of the timestamp sampling distributions. We further theoretically prove that CSE's asymptotic estimation error with respect to continuous-time risk is no greater than that of MSE. Finally, we construct a systematic benchmark covering synthetic, semi-synthetic, and eight real-world datasets to validate the effectiveness of CSE and systematically evaluate models' continuous-time predictive performance. Experiments show that CSE can recover continuous-time risk more accurately than MSE, while relying solely on MSE may not fully reflect models' continuous-time predictive performance in real-world scenarios. Our code can be obtained at https://github.com/hnu-vis/ITS-Bench.

时间序列评估指标不规则数据机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。