后训练让相同目标在不同模型层中产生不同计算依赖，揭示行为实现机制差异。
Same Targets, Different Computation: How Post-Training Divides Work Across Model Layers
- 设计四象限诊断法区分后期层变化对前期状态的依赖性。
- 数学指令下延迟层变化对目标得分提升达+3.43至+3.28逻辑值，交互接近零。
- 提示词与响应关系改变可使前期依赖上升5.56逻辑值，适用于模型微调研究者。
后训练中后期层的变化可能依赖于基础模型的早期状态，也可能仅依赖自身学习到的早期计算。我们通过一个四象限诊断框架，交叉对比基础或衍生的上游状态与基础或衍生的后期堆栈，以区分这两种情况。在数学提示下，OpenMath2 的后期堆栈在基础上游状态下使目标得分提升 +3.43 逻辑值，在自身上游状态下提升 +3.28 逻辑值，交互接近零。多个同源的 Llama-3.1-8B 指令遵循衍生模型表现出更强依赖性，而受控的代码与生物医学续写训练则在公共支持上接近零（-0.54 至 +2.20 逻辑值）。由于这些检查点存在多种差异，我们进一步隔离单一训练属性：两个 LoRA 微调学习相同的响应目标，但分别由熟悉的自然语言指令或新习得的伪码触发。仅改变这一提示-响应关系，即使前期依赖在 Qwen3-4B 上增加 +5.56 逻辑值、在 Llama-3.1-8B 上增加 +4.18 逻辑值，六次模型-种子组合均呈正向配对变化。所有五个测试的基线/指令对交互均为正值，且后期堆栈替换在约一半事件中改变了全词汇表 argmax。因此，后训练可使用不同上下文依赖结构实现相同目标行为。该诊断测量局部下一个词相容性，而非自由运行组件移植。
原文摘要 · Abstract (English)
A late-layer change learned during post-training may work on the base model's earlier state, or it may depend on earlier computation learned with it. We distinguish these cases with a four-cell diagnostic that crosses base or descendant upstream states with base or descendant late stacks. A large late-stack effect need not imply strong upstream dependence. On math prompts, OpenMath2's late stack changes the target margin by +3.43 logits after base upstream state and +3.28 after its own, giving a near-zero interaction. Several instruction-following descendants of the same Llama-3.1-8B base show greater dependence, while controlled code and biomedical continuation-training runs sit near zero on the common support; seven released descendants span -0.54 to +2.20 logits. Because those checkpoints differ in many ways, we then isolate one training property. Two LoRA fine-tunes learn identical target responses requested either by familiar natural-language instructions or by newly learned nonce codes. Changing only this cue-to-response relation increases upstream dependence by +5.56 logits on Qwen3-4B and +4.18 on Llama-3.1-8B, with a positive paired shift in all six model-by-seed runs. The interaction is also positive in all five released base/instruction pairs we test, and late-stack replacement changes the full-vocabulary argmax in about half of events. Post-training can therefore organize the same target behavior with different dependencies between earlier and later computation. The diagnostic measures local next-token compatibility, not free-running component transplantation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。