arXiv:2606.20547cs.LGcs.CV2026-06

将注意力令牌定义为矩阵李群元素,实现无需学习的几何感知注意力机制。

The Token Is a Group Element: On Lie-Algebra Attention over Matrix Lie Groups

  • 令牌直接作为李群元素,通过群对数计算相对姿态
  • 使用闭式李代数范数作为注意力得分,参数量减少50至80倍
  • 适用于带缩放剪切的非紧致非交换群,传统方法无法处理

本文提出一种新型注意力机制——李代数注意力(Lie-Algebra Attention),首次将注意力令牌视为矩阵李群元素而非向量。令牌为群元素 $g_i$,无特征负载,也无外部作用 $ρ(g)$。注意力得分基于相对姿态 $g_i^{-1} g_j$ 的李代数对数范数:$s_{ij} = -\ orm{\log(g_i^{-1} g_j)}_λ^2/τ$,采用块加权Frobenius内积,无需不可约表示、球谐函数或可学习核函数。该构造在任意包含相对姿态的对数坐标图上成立,涵盖非紧致非阿贝尔仿射群(如带尺度与剪切的Aff(2))。三组序列补全实验(在SE(2)、SO(3)、Aff(2)上)表明,该闭式得分性能优于同不变量下的可学习MLP核,在SE(2)上更优,且仅需50至80倍少的得分参数;而传统向量令牌基线因破坏不变性,误差高达五到十二个数量级。

原文摘要 · Abstract (English)

We place the attention token on the group: a token is an element $g_i$ of a matrix Lie group $G$ -- a bare transformation, with no feature payload and no external action $ρ(g)$ carrying it. To our knowledge this is the first attention construction whose tokens are bare matrix Lie group elements: their score is the closed-form algebra norm of the relative pose rather than a learned kernel, and it reaches the affine full-frame groups that every irrep- or surjective-exp-based method must exclude. We call it Lie-Algebra Attention. Once tokens are group elements, the rest follows with none of the usual representation-theoretic machinery. The relative geometry of a pair is canonical, $g_i^{-1} g_j$, so the pairwise invariant $w_{ij} = \log(g_i^{-1} g_j)$ is intrinsic rather than designed; equivariance under the diagonal $G$-action is tautological, and the cocycle condition holds automatically. The attention score is the negative squared algebra norm, $s_{ij} = -\|\log(g_i^{-1} g_j)\|_λ^2/τ$: the canonical proximity kernel under a block-weighted Frobenius inner product, with no irreducible representations, spherical harmonics, Clebsch-Gordan products, or learned kernel. The construction applies to any matrix Lie group on a chosen logarithm chart containing the relative poses, including the non-compact non-abelian affine groups with scale and shear that no vector-token attention method reaches: neither the irrep tradition nor surjective-exp methods. Three sequence-completion experiments, on SE(2), SO(3), and Aff(2), bear this out: the closed-form score matches a learned MLP kernel on the same invariant and outperforms it on SE(2), using 50 to 80x fewer score parameters, while a vector-token baseline breaks invariance by five to twelve orders of magnitude.

注意力机制李群几何深度学习群不变性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。