用相位几何替代注意力，实现高效长序列建模
The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle
- 将序列状态表示在单位圆上，通过相位偏移与傅里叶变换耦合
- 实现全局 $ ext{O}(N ext{log}N)$ 混合，参数量仅为自注意力的几分之一
- 适合需要高效建模振荡时序的场景，如信号处理、物理模拟
Transformer 模型虽重塑序列学习，但点积自注意力在长时序序列中存在二次复杂度瓶颈。本文提出相位变换器（Phasor Transformer）模块，将序列状态表示在单位圆流形 $S^1$ 上，结合轻量可训练相位偏移与无参离散傅里叶变换（DFT）进行令牌耦合，实现无需显式注意力图的全局 $ ext{O}(N ext{log}N)$ 混合。多个模块堆叠构成大相位模型（LPM）。在合成多频基准上的自回归时序预测任务中，该模型超越零参数持续基线，并随深度单调提升直至饱和；在参数量仅为自注意力几分之一的情况下，性能保持竞争力但未显著超越。结果确立了效率-精度边界，表明振荡域中的可扩展时间建模可通过几何约束的相位计算与确定性全局耦合实现。
原文摘要 · Abstract (English)
Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the Phasor Transformer block, a phase-native alternative representing sequence states on the unit-circle manifold $S^1$. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global $\mathcal{O}(N\log N)$ mixing without explicit attention maps. Stacking these blocks defines the Large Phasor Model (LPM). We validate LPM on autoregressive time-series prediction over synthetic multi-frequency benchmarks against honest baselines: it beats a zero-parameter persistence baseline and, with the corrected gradient path, improves monotonically with depth before saturating, while remaining competitive-but-not-superior to self-attention at a fraction of the parameter count. Our results establish an explicit efficiency--accuracy frontier, showing that scalable temporal modeling in oscillatory domains can emerge from geometry-constrained phase computation with deterministic global coupling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。