重新审视多步预测的偏差方差权衡,发现递归策略可能同时降低偏差、提高方差。
Epistemic Error Decomposition for Multi-step Time Series Forecasting: Rethinking Bias-Variance in Recursive and Direct Strategies
- 将多步预测误差分解为不可约噪声、结构近似误差和估计方差三部分
- 线性模型下递归策略无结构误差,非线性模型中递归可提升表达能力
- 递归策略的方差受雅可比放大因子影响,适用于高非线性或低噪声场景
多步时间序列预测常被认为:递归策略偏差高、方差低;直接策略偏差低、方差高。本文通过将期望误差分解为不可约噪声、结构近似差距和估计方差项,重新审视这一观点。对线性预测器,结构差距在任何数据集下恒为零;对非线性预测器,递归中重复组合可增强模型表达力,使结构差距依赖于模型与数据。进一步证明,递归策略在任意时序上的估计方差等于一步方差乘以基于雅可比的放大因子,反映参数误差敏感度。该视角解释了为何递归策略可能同时具有更低偏差和更高方差。在ETTm1数据集上使用多层感知机的实验验证了上述结论。结果为基于模型非线性程度与噪声特征选择递归或直接策略提供了实用指导,而非依赖传统偏差-方差直觉。
原文摘要 · Abstract (English)
Multi-step forecasting is often described through a simple rule of thumb: recursive strategies are said to have high bias and low variance, while direct strategies are said to have low bias and high variance. We revisit this belief by decomposing the expected multi-step forecast error into three parts: irreducible noise, a structural approximation gap, and an estimation-variance term. For linear predictors we show that the structural gap is identically zero for any dataset. For nonlinear predictors, however, the repeated composition used in recursion can increase model expressivity, making the structural gap depend on both the model and the data. We further show that the estimation variance of the recursive strategy at any horizon can be written as the one-step variance multiplied by a Jacobian-based amplification factor that measures how sensitive the composed predictor is to parameter error. This perspective explains when recursive forecasting may simultaneously have lower bias and higher variance than direct forecasting. Experiments with multilayer perceptrons on the ETTm1 dataset confirm these findings. The results offer practical guidance for choosing between recursive and direct strategies based on model nonlinearity and noise characteristics, rather than relying on traditional bias-variance intuition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。