arXiv:2512.18610cs.LG2025-12

点对点损失函数会引入无法消除的预测偏差,影响时间序列建模效果。

The Procrustean Bed of Time Series: The Optimization Bias in Point-wise Loss Functions

  • 用信息论量化点对点损失带来的系统性偏差,揭示其本质来源。
  • 偏差由序列长度和结构信噪比决定,与模型或优化器无关。
  • 提出新目标函数,在11个数据集上降低5%以上预测误差。

直观上,确定性越强的时间序列越易预测。然而,点对点损失函数(如MSE、MAE)独立评分每个时间戳,忽略时间依赖性,导致系统性优化偏差,无法通过提升模型表达力或优化器解决。本文定义期望优化偏差(EOB)为真实联合分布与点对点假设下的独立同分布近似之间的KL散度。在协方差平稳高斯假设下,推导出EOB的随机分量闭式表达,确立其为线性系统中不可消除的偏差下界,并通过高斯混合模型扩展至非线性情形。关键发现:该偏差由序列长度和结构信噪比(SSNR)内在决定,与具体模型、优化器或损失形式无关。据此提出基于序列长度缩减与结构正交化的去偏方案,结合DFT/DWT与新型谐调ℓ_p范数实现。大量实验验证了预测精度随SSNR与预测跨度的动态关系,解释了经典三角拟合失败为目标函数诱导的病理现象,并展现显著即插即用增益:在iTransformer上,跨11个数据集平均降低MSE/MAE 5.2%/5.0%,在9个数据集上缺失值修复任务中分别降低27.4%/19.4%。

原文摘要 · Abstract (English)

Intuitively, a more deterministic time series should be easier to forecast. However, point-wise loss functions (e.g., MSE and MAE), serving as differentiable surrogates for the ideal optimization target, score each timestamp independently and therefore disregard temporal dependence. This mismatch induces a systematic optimization bias that cannot be eliminated merely by improving model expressiveness or optimizer. To formalize this issue, we define the Expectation of Optimization Bias (EOB) as the Kullback--Leibler divergence between the true joint distribution and the factorized i.i.d. surrogate induced by the point-wise paradigm. Under covariance-stationary Gaussian assumptions, we derive closed-form expressions for the stochastic component of EOB, establishing it as an irreducible lower bound on the total bias in linear systems, and further extend it to nonlinear regimes through a Gaussian mixture model lower bound. Crucially, we prove this bias is governed intrinsically by two data properties, i.e., sequence length and Structural Signal-to-Noise Ratio (SSNR), regardless of specific model architecture, optimizer, or point-wise loss forms. This theory motivates a principled debiasing program based on sequence length reduction and structural orthogonalization, which we instantiate through DFT/DWT combined with a novel harmonized $\ell_p$ norm. Extensive experiments validate the predicted SSNR--horizon dynamics, resolve the classic trigonometric fitting failure as an objective-induced pathology, and demonstrate substantial plug-and-play gains. Notably, on iTransformer, our proposed objective reduces average MSE/MAE by 5.2%/5.0% in forecasting across 11 datasets and by 27.4%/19.4% in imputation across 9 datasets.

时间序列优化偏差损失函数信噪比

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。