指令微调模型在生成后期才稳定，晚层MLP是关键调节点。
The Convergence Gap: Instruction-Tuned Language Models Stabilize Later in the Forward Pass
- 通过层间分布距离度量，发现指令微调模型收敛更晚。
- 晚层MLP替换使预测差异增加0.34纳特，是最大调控点。
- 适合研究模型推理动态与微调机制的读者。
最终输出隐藏了检查点对下一词预测的确定时刻。我们提出收敛差距（convergence gap），一种基于模型差异的诊断方法，解码每一层的下一词分布，并测量其与模型自身最终分布的距离。在六组原始预训练与指令微调检查点的原生提示设置中，指令微调检查点在堆栈后期仍远离最终预测。该现象在终点匹配的原始与微调读出、无终点相同历史检验以及固定历史模板重放中均持续存在。匹配前缀干预识别出晚层MLP为最强调控点：将指令微调的晚层嫁接到预训练模型中，使晚期KL增加+0.34纳特；反之，将预训练的晚层换入指令微调模型中，减少-0.51纳特；而随机晚层扰动仅带来+0.003纳特，远低于真实嫁接的+0.327纳特。一个预选的Gemma案例研究提供了行为层面的合理性支持，但不作为基准主张。这些结果揭示了后训练阶段的一个稳健预测动态特征：释放的指令遵循检查点倾向于更晚收敛，且在相同历史条件下，晚层MLP计算是已测试中最强的双向控制机制。
原文摘要 · Abstract (English)
Final outputs hide when a checkpoint commits to its next-token prediction. We introduce the convergence gap, a model-diffing diagnostic that decodes each layer's next-token distribution and measures its distance to the model's own final distribution. Across six paired pretrained and instruction-tuned checkpoints in native prompting regimes, instruction-tuned checkpoints remain farther from their final predictions later into the stack. The effect persists under endpoint-matched raw and tuned readouts, endpoint-free same-history checks, and fixed-history template replay. Matched-prefix interventions identify late MLP windows as the largest tested leverage point: late IT grafts into PT hosts increase late KL by +0.34 nats, while PT-late swaps into IT hosts reduce it by -0.51 nats; matched random late perturbations give only +0.003 versus +0.327 for the true late graft. A preselected Gemma case study provides behavior-facing plausibility for the same late swap, without serving as a benchmark claim. These results identify a robust predictiondynamics signature of post-training: released instruction-following checkpoints tend to settle later, and late MLP computation is the strongest tested bidirectional handle on that delay under matched histories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。