arXiv:2606.01306cs.LGcs.IR2026-06KDD

提出新型Transformer模型,更好捕捉时间序列中的高频变化。

FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

论文配图:FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting
图 1 · 摘自论文原文
  • 用反向注意力机制修复自注意力的低通滤波缺陷
  • 动态调制频带能量,适应随时间变化的频率特征
  • 在多个基准上超越现有方法,计算开销小

尽管基于Transformer的架构已成为多变量时间序列预测(MTSF)的主流范式,但其核心自注意力机制本质上是低通滤波器,会系统性平滑掉对局部剧烈变化至关重要的高频信号。近期进展已逐步引入频域操作以缓解这一偏差,但多数现有设计依赖固定谱基并采用全局统一的调制方式,隐含假设频率响应随时间不变。这忽略了真实序列的谱特性常随时间演化的重要特性,导致均匀调制难以捕捉精细的时间动态。为此,我们提出FAiT:一种频域感知的反向Transformer。具体而言,FAiT通过反向注意力机制内部修正谱偏差,将注意力图视为可学习的低通算子,并通过反转注意力矩阵构建专用的互补高通分支,以恢复被抑制的瞬态信号。此外,FAiT引入动态时空调制(DTFM),合成实例相关的权重,自适应地重新校准各频带能量,实现对演化多尺度模式的精细控制。在广泛使用的基准上的大量实验表明,FAiT持续优于最先进的Transformer及频域增强基线,同时保持计算效率。

原文摘要 · Abstract (English)

While Transformer-based architectures have established themselves as a dominant paradigm in Multivariate Time Series Forecasting (MTSF), their core self-attention mechanism inherently functions as a low-pass filter, systematically smoothing out high-frequency signals vital for sharp local changes. Recent advancements have increasingly incorporated frequency-domain operations to address this bias, however, most existing designs rely on fixed spectral bases and apply sequence-wise (uniform) modulation, implicitly assuming a time-invariant frequency response. This overlooks a key property of real-world series that their spectral characteristics often evolve over time, making uniform modulation insufficient for capturing fine-grained temporal dynamics. To tackle these limitations, we propose FAiT, a Frequency-Aware inverted Transformer. Specifically, FAiT rectifies the spectral bias internally through Inverted Attention, which interprets the attention map as a learnable low-pass operator and constructs a dedicated complementary high-pass branch by inverting the attention matrix to recover attenuated transient signals. Furthermore, FAiT introduces Dynamic Temporal-Frequency Modulation (DTFM), which synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands, enabling fine-grained control over evolving multi-scale patterns. Extensive experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines, while maintaining computational efficiency.

时间序列Transformer频域分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。