通过自适应掩码损失,让模型学会忽略冗余历史数据,提升预测准确率。
Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
- 用动态掩码识别关键时间片段,引导模型聚焦有效信号。
- 在多个数据集上显著降低误差,最高提升12.3%预测精度。
- 适合需要高鲁棒性时序预测的场景,如金融与能源管理。
时间序列预测在能源管理和金融市场等关键领域至关重要。尽管基于深度学习的方法(如MLP、RNN、Transformer)已取得显著进展,但主流的“长序列信息增益假设”存在内在局限。系统实验揭示了一种反直觉现象:适度截断历史数据反而能提升预测准确性,表明现有模型在训练中学习了大量冗余特征(如噪声或无关波动),从而影响有效信号提取。基于信息瓶颈理论,本文提出一种新方法——自适应掩码损失与表示一致性(AMRC),包含两个核心组件:1)动态掩码损失,自适应识别具有判别性的时序段,指导训练过程中的梯度下降;2)表示一致性约束,稳定输入、标签与预测之间的映射关系。实验结果表明,AMRC能有效抑制冗余特征学习,并显著提升模型性能。该工作不仅挑战了传统时间建模假设,还为构建高效稳健的预测模型提供了新的理论见解与方法突破。
原文摘要 · Abstract (English)
Time series forecasting plays a pivotal role in critical domains such as energy management and financial markets. Although deep learning-based approaches (e.g., MLP, RNN, Transformer) have achieved remarkable progress, the prevailing "long-sequence information gain hypothesis" exhibits inherent limitations. Through systematic experimentation, this study reveals a counterintuitive phenomenon: appropriately truncating historical data can paradoxically enhance prediction accuracy, indicating that existing models learn substantial redundant features (e.g., noise or irrelevant fluctuations) during training, thereby compromising effective signal extraction. Building upon information bottleneck theory, we propose an innovative solution termed Adaptive Masking Loss with Representation Consistency (AMRC), which features two core components: 1) Dynamic masking loss, which adaptively identified highly discriminative temporal segments to guide gradient descent during model training; 2) Representation consistency constraint, which stabilized the mapping relationships among inputs, labels, and predictions. Experimental results demonstrate that AMRC effectively suppresses redundant feature learning while significantly improving model performance. This work not only challenges conventional assumptions in temporal modeling but also provides novel theoretical insights and methodological breakthroughs for developing efficient and robust forecasting models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。