提出高效分层自回归变压器,兼顾速度、精度与长期预测能力。
AutoHFormer: Efficient Hierarchical Autoregressive Transformer for Time Series Prediction
- 分层处理时序片段并行计算,再逐段细化,提升效率同时保持时间因果性。
- 动态窗口注意力使训练快10.76倍、内存减少6.06倍,长程预测更准。
- 适合需要快速高精度时序预测的工业场景,如能源调度与金融分析。
时序预测需同时满足三大目标:(1)严格的时间因果性以保证预测可靠性;(2)亚二次复杂度以实现实际可扩展性;(3)多尺度模式识别以支持准确的长程预测。我们提出 AutoHFormer,一种分层自回归变压器,通过三项创新解决上述挑战:(1)分层时间建模:将预测分解为可并行处理的片段级块,随后进行片段内顺序优化,双重尺度设计既保持时间连贯性又提升计算效率;(2)动态窗口注意力:采用可学习的因果窗口与指数衰减机制,降低复杂度的同时保留精确的时间关系,避免标准 Transformer 的反因果问题及 RNN 混合模型的串行瓶颈;(3)自适应时间编码:结合固定振荡模式捕捉短期波动与可学习衰减速率建模长期趋势,实现多尺度时间特征捕捉。大规模实验表明,在 PEMS08 数据集上,AutoHFormer 相比 PatchTST 实现 10.76 倍训练加速与 6.06 倍内存节省,且在 96–720 步长程预测中多数情况下保持稳定精度。该工作树立了高效精准时序建模的新基准。方法与所有基线的实现代码已开源于 https://github.com/lizzyhku/Autotime。
原文摘要 · Abstract (English)
Time series forecasting requires architectures that simultaneously achieve three competing objectives: (1) strict temporal causality for reliable predictions, (2) sub-quadratic complexity for practical scalability, and (3) multi-scale pattern recognition for accurate long-horizon forecasting. We introduce AutoHFormer, a hierarchical autoregressive transformer that addresses these challenges through three key innovations: 1) Hierarchical Temporal Modeling: Our architecture decomposes predictions into segment-level blocks processed in parallel, followed by intra-segment sequential refinement. This dual-scale approach maintains temporal coherence while enabling efficient computation. 2) Dynamic Windowed Attention: The attention mechanism employs learnable causal windows with exponential decay, reducing complexity while preserving precise temporal relationships. This design avoids both the anti-causal violations of standard transformers and the sequential bottlenecks of RNN hybrids. 3) Adaptive Temporal Encoding: a novel position encoding system is adopted to capture time patterns at multiple scales. It combines fixed oscillating patterns for short-term variations with learnable decay rates for long-term trends. Comprehensive experiments demonstrate that AutoHFormer 10.76X faster training and 6.06X memory reduction compared to PatchTST on PEMS08, while maintaining consistent accuracy across 96-720 step horizons in most of cases. These breakthroughs establish new benchmarks for efficient and precise time series modeling. Implementations of our method and all baselines in hierarchical autoregressive mechanism are available at https://github.com/lizzyhku/Autotime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。