PENGUIN通过周期嵌套分组注意力提升长时序预测性能
PENGUIN: Enhancing Transformer with Periodic-Nested Group Attention for Long-term Time Series Forecasting
- 引入周期感知相对偏置和分组多查询注意力机制
- 在多个基准上优于主流MLP与Transformer模型
- 适合处理含日、周等多重周期的时序数据
尽管Transformer架构取得进展，其在长时序预测（LTSF）中的有效性仍存争议。本文研究将显式周期性建模融入自注意力机制以提升Transformer在LTSF中的表现。提出PENGUIN，一种简单而有效的周期嵌套分组注意力机制。该方法引入周期感知相对注意力偏置，直接捕捉周期结构，并采用分组多查询注意力机制，处理时间序列中同时存在的多种周期性（如日周期与周周期）。在多个不同基准上的大量实验表明，PENGUIN持续优于基于MLP和Transformer的模型。代码已公开于https://github.com/ysygMhdxw/AISTATS2026_PENGUIN。
原文摘要 · Abstract (English)
Despite advances in the Transformer architecture, their effectiveness for long-term time series forecasting (LTSF) remains controversial. In this paper, we investigate the potential of integrating explicit periodicity modeling into the self-attention mechanism to enhance the performance of Transformer-based architectures for LTSF. Specifically, we propose PENGUIN, a simple yet effective periodic-nested group attention mechanism. Our approach introduces a periodic-aware relative attention bias to directly capture periodic structures and a grouped multi-query attention mechanism to handle multiple coexisting periodicities (e.g., daily and weekly cycles) within time series data. Extensive experiments across diverse benchmarks demonstrate that PENGUIN consistently outperforms both MLP-based and Transformer-based models. Code is available at https://github.com/ysygMhdxw/AISTATS2026_PENGUIN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。