Transformer结构可由极坐标状态估计自然推导，解释其设计原理。
The Transformer as a Polar State Estimator
- 用极坐标建模隐状态，分离径向与球面动态
- 归一化约束球面，注意力聚合方向证据，残差连接实现增量更新
- 新模型保留几何修正项，更符合物理意义，适合理论研究者
我们证明，Transformer的核心组件——注意力机制、残差连接和归一化——均源于单一的几何状态估计问题。通过在极坐标下建模隐状态，天然分离了径向与超球面动态，从而得到一种精度加权的滤波过程：归一化强制超球面约束，注意力聚合方向性证据，残差连接实现增量状态更新。在舍弃几何修正项后，标准Transformer块（带旋转位置编码）被重新恢复，表明其架构源自底层估计问题，而非独立设计选择。所提出的【极坐标Transformer】保留这些几何修正项。
原文摘要 · Abstract (English)
We show that the core components of the Transformer---attention, residual connections, and normalization---arise naturally from a single geometric state estimation problem. Modeling the latent state in polar coordinates naturally separates radial and hyperspherical dynamics, yielding a precision-weighted filtering procedure in which normalization enforces the hyperspherical constraint, attention aggregates directional evidence, and the residual connection implements an incremental state update. The standard Transformer block with rotary positional encodings is recovered by discarding the geometric correction terms of the resulting state estimator, showing that its architecture follows from the underlying estimation problem rather than from independent design choices. The proposed \textit{Polar Transformer} retains these geometric corrections.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。