用注意力机制优化经验回放,让时间序列模型持续学习不遗忘。
Attention-based Experience Replay Framework for Continual Learning of Agnostic Time Series Forecasting Models
- 引入注意力驱动的经验回放,动态选择关键历史数据
- 在多个数据集上保持长期预测精度,降低重训练成本
- 适合需要长期部署的动态环境预测任务
深度学习在人工智能领域取得显著进展,尤其在机器人、图像和声音处理方面。然而,神经网络仍严重依赖大规模静态数据集,而现实场景中数据分布常随时间演变。持续学习旨在开发能增量适应新数据、同时保持稳定与可塑性平衡的模型。本文提出一种新型持续时间序列预测框架,通过注意力机制引导的经验回放策略,扩展现有静态预测模型,使其在动态环境中持续学习并保留旧知识,有效缓解灾难性遗忘。该框架在标准预测基准及具有多样时序行为的压电井水位数据集上评估,结果表明其能维持或提升长期预测性能,同时减少重训练开销与数据需求,有利于在动态真实场景中部署预测模型。
原文摘要 · Abstract (English)
Deep learning has led to remarkable progress in artificial intelligence, particularly in robotics, imaging and sound processing. However, a major limitation of neural networks remains their strong dependence on large and stationary datasets. In many real-world applications, these conditions are rarely met due to evolving and dynamic environments where data distributions change over time. Continual learning aims to address this challenge by developing models capable of adapting incrementally while maintaining a balance between stability and plasticity under computational constraints. In this work, we introduce a novel framework for continual time series forecasting, designed to extend existing static forecasting models commonly used in the literature by incorporating an Experience Replay strategy guided by Attention mechanisms. This approach allows the model to adapt dynamically to new contexts while preserving prior knowledge, effectively mitigating catastrophic forgetting. The framework is evaluated on standard forecasting benchmarks as well as on a piezometric dataset exhibiting diverse temporal behaviors. Results show that our approach effectively increases or maintains predictive performance over time while reducing retraining costs and data requirements, thus facilitating the deployment of forecasting models in dynamic and real-world settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。