通过动态替换低质量时间片段,提升模型在噪声数据下的预测鲁棒性。
SEER: Transformer-based Robust Time Series Forecasting via Automated Patch Enhancement and Replacement
- 用MoE和通道自适应机制增强片段表征,提升信息质量。
- 两阶段修复:先过滤低质片段,再用全局特征替代并优化。
- 适合处理含缺失、异常等噪声的真实时间序列数据。
时间序列预测在诸多需精准决策的领域中至关重要。分块技术能有效捕捉时序依赖,但现有方法通常静态使用全部分块,无法应对真实数据中的缺失值、分布漂移、异常点和白噪声等质量问题。本文提出一种鲁棒的时间序列预测框架SEER:首先设计增强嵌入模块,利用混合专家(MoE)架构优化片段级表征,并通过通道自适应感知机制获取序列级标记表示;其次引入可学习分块替换模块,包含两阶段过程:1)动态过滤机制剔除负向片段标记;2)替换注意力模块用全局序列标记替代低质片段,并通过因果注意力进一步优化其表示。大量实验表明,SEER在多个基准上达到最先进性能。
原文摘要 · Abstract (English)
Time series forecasting is important in many fields that require accurate predictions for decision-making. Patching techniques, commonly used and effective in time series modeling, help capture temporal dependencies by dividing the data into patches. However, existing patch-based methods fail to dynamically select patches and typically use all patches during the prediction process. In real-world time series, there are often low-quality issues during data collection, such as missing values, distribution shifts, anomalies and white noise, which may cause some patches to contain low-quality information, negatively impacting the prediction results. To address this issue, this study proposes a robust time series forecasting framework called SEER. Firstly, we propose an Augmented Embedding Module, which improves patch-wise representations using a Mixture-of-Experts (MoE) architecture and obtains series-wise token representations through a channel-adaptive perception mechanism. Secondly, we introduce a Learnable Patch Replacement Module, which enhances forecasting robustness and model accuracy through a two-stage process: 1) a dynamic filtering mechanism eliminates negative patch-wise tokens; 2) a replaced attention module substitutes the identified low-quality patches with global series-wise token, further refining their representations through a causal attention mechanism. Comprehensive experimental results demonstrate the SOTA performance of SEER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。