Transformer在金融时间序列预测中会因噪声重用导致误差增大。
Forecast collapse of transformer-based models under squared loss in financial time series
- 理论分析指出，金融数据条件期望退化为常数，模型越复杂越不稳定。
- 实验证明，Transformer在高频汇率数据上多数窗口误差高于线性模型。
- 适合关注模型过拟合与金融预测失效机制的研究者阅读。
我们研究了在平方损失下，对弱条件结构的时间序列进行轨迹预测，采用高表达力的预测模型。基于经典平方损失风险最小化的刻画，强调了条件期望有效退化的区域，导致贝叶斯最优预测器趋于平凡（价格恒定、收益为零）。在此区域，模型表达力增强不会提升预测精度，反而引入围绕最优预测器的虚假轨迹波动。这些波动源于噪声重复利用，导致预测方差上升而偏差无减。这为Transformer在金融时间序列上的预测退化提供了过程级解释。我们在高频欧元/美元汇率数据上进行了数值实验，分析轨迹级预测误差分布。结果显示，多数预测窗口中，Transformer模型的误差显著大于简单线性基准，与理论揭示的方差驱动机制一致。
原文摘要 · Abstract (English)
We study trajectory forecasting under squared loss for time series with weak conditional structure, using highly expressive prediction models. Building on the classical characterization of squared-loss risk minimization, we emphasize regimes in which the conditional expectation of future trajectories is effectively degenerate, leading to trivial Bayes-optimal predictors (flat for prices and zero for returns in standard financial settings). In this regime, increased model expressivity does not improve predictive accuracy but instead introduces spurious trajectory fluctuations around the optimal predictor. These fluctuations arise from the reuse of noise and result in increased prediction variance without any reduction in bias. This provides a process-level explanation for the degradation of Transformerbased forecasts on financial time series. We complement these theoretical results with numerical experiments on high-frequency EUR/USD exchange rate data, analyzing the distribution of trajectory-level forecasting errors. The results show that Transformer-based models yield larger errors than a simple linear benchmark on a large majority of forecasting windows, consistent with the variance-driven mechanism identified by the theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。