arXiv:2606.18367cs.LG2026-06中稿 · the Workshop on Fo…被引 6

交通速度预测中,模型在拥堵转换期表现骤降,但传统评估掩盖了这一问题。

Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecasting

  • 按交通状态分组评估,暴露模型在拥堵转换期的严重失效
  • 转换期平均误差达11英里/小时(整体仅3英里/小时),预测区间覆盖率低至55%
  • 提出混合增强方法,兼顾准确率与关键时段预测可靠性

标准基准使用综合指标评估时间序列基础模型(TSFMs),但可能掩盖关键运行状态下的严重失败。我们引入按状态分层的评估方法,应用于两个主流交通速度基准上的三种TSFMs。交通在自由流与拥堵状态间存在突变,过渡期呈现双峰速度分布。按交通状态分组后,准确率与预测区间覆盖率在转换期显著下降:转换期平均绝对误差(MAE)达11英里/小时(整体为3英里/小时),90%预测区间的实际覆盖率达最低55%。这些失败在总体指标下被掩盖,因自由流数据占样本主导。一种简单的历史条件基线(从各传感器训练分布采样)在转换期覆盖率优于所有TSFM,但整体准确率差得多。我们提出双峰混合增强(BMA),通过融合TSFM预测与历史分布知识,在保持TSFM高准确率的同时,接近历史基线的转换期覆盖率。结果表明,TSFM评估应引入状态感知机制,以揭示被综合指标隐藏的故障。

原文摘要 · Abstract (English)

Standard benchmarks evaluate time series foundation models (TSFMs) using aggregate metrics, but these can mask severe failures in critical operating regimes. We introduce regime-stratified evaluation and apply it to three TSFMs on two standard traffic speed benchmarks. Traffic exhibits abrupt regime switching between free-flow and congested states, producing bimodal speed distributions during transitions. When we stratify by traffic regime, both accuracy and prediction-interval coverage degrade sharply during transitions: transition-regime MAE reaches 11 mph (versus 3 mph overall), and empirical coverage of 90% prediction intervals drops as low as 55%. These failures are invisible in aggregate metrics because free-flow observations dominate the sample. A simple historical conditional baseline (sampling from per-sensor training distributions) achieves better transition coverage than any TSFM, but has far worse overall accuracy. We propose bimodal mixture augmentation (BMA), a post-hoc method that combines TSFM forecasts with historical distributional knowledge, approaching the historical baseline's transition coverage while preserving the TSFM's accuracy. Our results suggest that TSFM benchmarks should incorporate regime-aware evaluation to surface failures that aggregate metrics hide.

时间序列交通预测模型评估状态切换

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。