用小波分解增强生成样本,让预训练时序模型持续学习不遗忘。
R-Tuning: Wavelet-Decomposed Replay and Semantic Alignment for Continual Adaptation of Pretrained Time-Series Models
- 通过小波分解多频带生成趋势保持的合成数据,提升知识回放效率
- 在新任务上降低MAE和MSE超46%,旧任务性能提升5.7%~6.0%
- 适合资源受限场景下少样本持续学习,尤其对时序预测应用有价值
预训练模型在时序预测中表现出色,但适应动态数据分布仍面临挑战。主要瓶颈在于无法获取原始训练数据,仅用新数据微调易导致灾难性遗忘。为此,我们提出R-Tuning框架,用于预训练时序模型的持续适应。该方法通过频率感知的回放策略构建统一潜在空间,融合小波分解生成的多频带趋势保持样本,增强表示多样性与回放效率;同时引入潜在一致性约束,将新样本表示对齐到历史任务空间,实现紧凑且语义一致的联合优化,保障知识保留与适应能力。大量实验表明,R-Tuning在新任务上可使MAE和MSE分别降低46.9%和46.8%,旧任务性能提升5.7%和6.0%。即使合成样本仅占新数据集5%,在少样本设置下仍优于所有现有基线。
原文摘要 · Abstract (English)
Pre-trained models have demonstrated exceptional generalization capabilities in time-series forecasting; however, adapting them to evolving data distributions remains a significant challenge. A key hurdle lies in accessing the original training data, as fine-tuning solely on new data often leads to catastrophic forgetting. To address this issue, we propose Replay Tuning (R-Tuning), a novel framework designed for the continual adaptation of pre-trained time-series models. R-Tuning constructs a unified latent space that captures both prior and current task knowledge through a frequency-aware replay strategy. Specifically, it augments model-generated samples via wavelet-based decomposition across multiple frequency bands, generating trend-preserving and fusion-enhanced variants to improve representation diversity and replay efficiency. To further reduce reliance on synthetic samples, R-Tuning introduces a latent consistency constraint that aligns new representations with the prior task space. This constraint guides joint optimization within a compact and semantically coherent latent space, ensuring robust knowledge retention and adaptation. Extensive experimental results demonstrate the superiority of R-Tuning, which reduces MAE and MSE by up to 46.9% and 46.8%, respectively, on new tasks, while preserving prior knowledge with gains of up to 5.7% and 6.0% on old tasks. Notably, under few-shot settings, R-Tuning outperforms all state-of-the-art baselines even when synthetic proxy samples account for only 5% of the new task dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。