通过几何结构追踪,发现模型间可恢复的隐层模式。
ATLAS: Constitution-Conditioned Latent Geometry and Redistribution Across Language Models and Neural Perturbation Data
- 以局部图表为单位,分析宪法条件引发的表征几何变化。
- 在多个模型与数据中,检测到 310/320 条源数据一致响应,AUC 达 0.984。
- 适合研究模型迁移、神经扰动与隐空间结构可复现性的读者。
宪法条件下的后训练可被看作对模型学习表征几何的结构性扰动。我们提出 ATLAS,一种以几何为核心的分析框架,用于追踪宪法诱导的隐藏状态结构在不同模型与数据基上的演变。不同于将相关单元视为单一行为、神经元、向量或补丁,ATLAS 测试一个局部图表,其切线结构、占据分布和行为耦合可在系统变化下测量。在 Gemma 模型上,锚定的源局部图表捕获了 310 / 320 个已审查源行和全部 84 / 84 个评分翻转行,但精确补丁充分性未闭合,因此可导出的单元是更广泛的源定义家族。冻结该家族后,在未适应的 Phi 模型中重新识别出目标局部实现,完全裁定的确认性对比分离达到 AUC 0.984,平均差距 5.50。在保留的 ALM8 小鼠前额叶皮层扰动数据中,同一源定义家族在 5/5 折中获得支持,保留平均 AUC 0.72,平均折差 4.50。多选分析揭示主要边界:目标局部信号可出现而无需源忠实闭合。最终对应关系并非坐标身份、位置身份或目标侧中介定理,而是几何重复下的再分配:写入宪法可诱导可恢复的潜在几何,其组织结构在模型与基底变化中仍可检测,尽管局部坐标、占据与行为表达发生转移。
原文摘要 · Abstract (English)
Constitution-conditioned post-training can be analysed as a structured perturbation of a model's learned representational geometry. We introduce ATLAS, a geometry-first program that traces constitution-induced hidden-state structure across charts, models, and substrates. Instead of treating the relevant unit as a single behaviour, neuron, vector, or patch, ATLAS tests a local chart whose tangent structure, occupancy distribution, and behavioural coupling can be measured under system change. On Gemma, the anchored source-local chart captures 310 / 320 reviewed source rows and all 84 / 84 reviewed score-flip rows, but compact exact-patch sufficiency does not close, so the exportable unit is the broader source-defined family. Freezing that family, we re-identify a target-local realisation in an unadapted Phi model, where the fully adjudicated confirmatory contrast separates with AUC 0.984 and mean gap 5.50. In held-out ALM8 mouse frontal-cortex perturbation data, the same source-defined family receives support across 5/5 folds, with mean held-out AUC 0.72 and mean fold gap 4.50. A multiple-choice analysis provides the main boundary: nearby target-local signals can appear without source-faithful closure. The resulting correspondence is not coordinate identity, site identity, or a target-side mediation theorem. It is geometric recurrence under redistribution: written constitutions can induce recoverable latent geometry whose organisation remains detectable across model and substrate changes while its local coordinates, occupancy, and behavioural expression shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。