位置编码决定神经网络权重中能读出的对称性,影响结构可解释性。
Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weights

- 通过可观测性层级揭示对称性读取受编码和观测方式共同影响
- 实验显示不同位置编码对不同旋转对称性的读出能力差异显著
- 为权重级结构分析提供了基于可观测性的设计原则
对训练后神经网络权重的后验分析常试图直接从参数中恢复几何结构。我们发现,对于带位置编码的神经场,权重中可见的对称性并非真实对称群本身,而是由训练参数、位置编码(PE)和读出可观测量共同决定的可观测对称集。通过构建精确可观测性层级 $G_{\mathrm{obs}}^{\mathrm{exact}} \subseteq G_{\mathrm{lift}}^{\mathrm{exact}}(ϕ) \cap G_{\mathrm{true}}$,表明即使目标函数具有几何对称性,若位置编码无法表示对应变换,该对称性在权重层面也可能不可见。我们在二维符号距离函数上使用多组形状对称群、位置编码与基于格拉姆的可观测量进行测试,结果一致显示:DyadicAxisPE支持 $D_4$ 敏感读出但结构性抑制 $D_3$ 旋转;TriAxisPE 在三个120度分离轴替代坐标轴后,在测试的格拉姆可观测量下降低 $D_3$ / $D_6$ 读出得分;随机傅里叶特征主要表现出 $π$-旋转响应。这些发现表明,位置编码设计不仅影响逼近行为,也决定了哪些结构可通过后验权重读出。这为可解释性分析提供了可观测性依赖的理论基础。
原文摘要 · Abstract (English)
Post-hoc analysis of trained neural network weights often seeks to recover geometric structure directly from the parameters. We show that, for positional-encoding-equipped neural fields, the symmetry visible from weights is not the true symmetry group itself, but an observable symmetry set determined by the trained parameters, the positional encoding (PE), and readout observable. We formulate this dependence through an exact observability hierarchy, $G_{\mathrm{obs}}^{\mathrm{exact}} \subseteq G_{\mathrm{lift}}^{\mathrm{exact}}(ϕ) \cap G_{\mathrm{true}}$, where $G_{\mathrm{lift}}^{\mathrm{exact}}(ϕ)$ is the set of input transformations that the PE can exactly lift to the feature space. The hierarchy implies that even when a target function has a geometric symmetry, that symmetry may be structurally invisible to weight-level observables if the PE does not represent the corresponding transformation. We test this prediction using MLPs trained on two-dimensional signed distance functions with multiple shape symmetry groups, positional encodings, and Gram-based observables. The results show a consistent PE-dependent pattern: DyadicAxisPE supports $D_4$-sensitive readout but structurally suppresses $D_3$ rotations, TriAxisPE yields lower $D_3$ / $D_6$ readout scores under the tested Gram observables by replacing coordinate axes with three 120-degree-separated axes, and random Fourier features mainly exhibit a $π$-rotation response under these readouts. These findings show that PE design affects not only approximation behavior but also which structures are accessible to post-hoc weight-level readouts. This provides a basis for a principled observable-dependent symmetry readout.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。