主动特征平面的全同性反而更低,挑战了语义集中于活跃特征的假设。
Do Active SAE Feature Planes Carry More Holonomy? A Preregistered Reversal in Gemma
- 用雅可比传输法测量残差流中局部环路的全同性变化
- 主动特征平面的全同性比混合特征平面低,调整后对数对比为-0.294
- 结果为可审计的反向发现,适用于研究模型内在几何结构者
本文检验了在Gemma 2 2B中,主动稀疏自编码器(SAE)特征平面是否集中了更多全同性,这是对语义集中预测的具体操作化。全同性在第12层到第13层残差流读出层通过沿小环路携带局部参照系并使用受限雅可比传输规则进行测量,结果旋转量按包围面积归一化。实验设计、材料阈值、分析与结论规则均预先注册并冻结。预测被反向证伪:主动特征平面的全同性低于匹配的混合特征控制组,调整后对数对比为-0.29439,95%置信区间[-0.43989, -0.14889]。仅基于幅度的解释在此设计中不成立;在匹配幅度下,随机、混合与主动特征平面的三重排序因共支持失败而未定义。冻结后的诊断在相同读出层验证了小样本上的面积律,配对回归下匹配中心位移受控,并识别出传输畸变为潜在机制或混杂因素。因此,该结果为狭窄、可审计的操作性反转,而非因果声明表明意义抑制全同性。原因仍开放,激活强度几何、特征参与程度、字典几何、匹配中心位移、激活流形邻近度及传输剪切均为待检验的候选因素。
原文摘要 · Abstract (English)
This paper tests whether holonomy concentrates on active sparse-autoencoder (SAE) feature planes in Gemma 2 2B, a concrete operationalization of the broader semantic-concentration prediction. Holonomy is measured at the final-token layer-12 to layer-13 residual-stream readout by carrying a local frame around small loops using the instrument's restricted-Jacobian transport rule, then normalizing the resulting rotation by enclosed area. The design, materiality threshold, analysis, and verdict rules were preregistered and frozen before the analysed measurements were inspected. The prediction was falsified in reverse: active-feature planes carried less holonomy than matched mixed-feature controls, with an adjusted log contrast of -0.29439 and 95% interval [-0.43989, -0.14889]. A magnitude-only explanation was not supported in this design, while the three-way ordering across random, mixed-feature, and active-feature planes was undefined at matched magnitude because common support failed. Post-freeze diagnostics at the same readout supported the area law on a small validation subset, bounded matched-center displacement under a simple paired regression, and identified transport distortion as a live mechanism or confound. The result is therefore a narrow, auditable operational reversal, not a causal claim that meaning suppresses holonomy. The cause remains open, with activation-strength geometry, degree of feature engagement, dictionary geometry, matched-center displacement, activation-manifold proximity, and transport shear as live alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。