让大模型晚层表示直接参与训练，提升推理能力
Tuning Language Models by Mixture-of-Depths Ensemble
- 用可学习路由权重融合多个晚层输出，形成新训练信号
- 在少量参数开销下，提升模型推理性能，媲美全层微调
- 适合希望轻量优化大模型推理能力的研究者
基于Transformer的大语言模型传统上依赖最终层损失进行微调，以及最终层表示进行预测，可能忽略了深层中蕴含的预测信息。解释工具如logit lens表明，深层表示已包含高度成型的任务相关预测。本文探讨能否将此观察转化为有效训练信号。研究发现，聚焦于深层训练可获得与最终层相当的损失，且测试时表现互补。基于此，提出一种名为混合深度集成（Mixture-of-Depths Ensemble, MoDE）的微调框架，将深层视为一个通过可学习路由权重贡献最终logits的集成系统。MoDE可叠加于任何现有微调方法（如LoRA），实验显示其以极小参数开销小幅提升推理性能。本工作验证了深层logits可直接用于训练，并能替代更大可训练模块实现相近效果。
原文摘要 · Abstract (English)
Transformer-based Large Language Models (LLMs) traditionally rely on final-layer loss for finetuning and final-layer representations for predictions, potentially overlooking the predictive power embedded in late layers. Interpretability tools such as the logit lens show that late-layer representations already carry largely formed, task-relevant predictions; here we ask whether that observation can be turned into an actionable training signal. We find that focusing tuning effort on these layers can yield losses comparable to those of the final layer, with complementary test-time behaviour. Building on this, we introduce a tuning framework, Mixture-of-Depths Ensemble (MoDE), which treats the late layers as an ensemble that contributes to the final logits through learned routing weights. MoDE can be applied on top of any existing tuning method (e.g., LoRA) and, in our experiments, modestly improves reasoning performance at a small parameter overhead. We present MoDE as a mechanism study showing that late-layer logits can be made directly useful for tuning, and that they can substitute for substantially larger trainable modules with comparable performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。