改进Adam优化器，让时间序列预测更适应数据分布变化。
Rethinking Adam for Time Series Forecasting: A Simple Heuristic to Improve Optimization under Distribution Shifts
- 移除Adam的二阶偏差校正，提升对动态损失景观的响应速度。
- 在ETT数据集上，比Adam平均降低12.8%的MSE和5.7%的MAE。
- 无需额外超参数，可无缝集成到现有模型中，适合实际应用。
时间序列预测常受非平稳性影响，尤其是数据分布随时间漂移。这种动态特性会削弱自适应优化器（如Adam）的性能，因其通常针对平稳目标设计。本文重新审视了在非平稳预测中使用Adam的问题，发现其二阶偏差校正限制了对变化损失景观的响应能力。为此，提出轻量级变体TS_Adam，通过移除学习率计算中的二阶修正来提升适应性，同时保留优化器核心结构且不引入额外超参数。该方法易于集成至现有模型，在长短期预测任务中均表现更优。在使用MICN模型的ETT数据集上，相比Adam，TS_Adam平均降低12.8%的MSE和5.7%的MAE。结果表明，TS_Adam是一种实用且通用的非平稳数据优化策略。代码已开源：https://github.com/DD-459-1/TS_Adam。
原文摘要 · Abstract (English)
Time-series forecasting often faces challenges from non-stationarity, particularly distributional drift, where the data distribution evolves over time. This dynamic behavior can undermine the effectiveness of adaptive optimizers, such as Adam, which are typically designed for stationary objectives. In this paper, we revisit Adam in the context of non-stationary forecasting and identify that its second-order bias correction limits responsiveness to shifting loss landscapes. To address this, we propose TS_Adam, a lightweight variant that removes the second-order correction from the learning rate computation. This simple modification improves adaptability to distributional drift while preserving the optimizer core structure and requiring no additional hyperparameters. TS_Adam integrates easily into existing models and consistently improves performance across long- and short-term forecasting tasks. On the ETT datasets with the MICN model, it achieves an average reduction of 12.8% in MSE and 5.7% in MAE compared to Adam. These results underscore the practicality and versatility of TS_Adam as an effective optimization strategy for real-world forecasting scenarios involving non-stationary data. Code is available at: https://github.com/DD-459-1/TS_Adam.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。