通过门控机制实现大模型逐令牌跳过计算，节省算力同时保持高精度。
What Layers When: Learning to Skip Compute in LLMs with Residual Gates
- 用可微分的门控单元控制每层注意力与MLP输出，决定是否跳过计算。
- 长文本推理中节省15%算力，准确率仍超基线90%；大模型下优势更明显。
- 适合追求高效推理的开发者，兼容量化、剪枝等压缩技术。
我们提出GateSkip，一种简单的残差流门控机制，可在仅解码器的语言模型中实现逐令牌的层跳过。每个注意力/MLP分支配备一个sigmoid-linear门控单元，在输出重新进入残差流前进行压缩。推理时，按门控值对令牌排序，并根据每层预算跳过低重要性部分。相比早期退出或路由型混合深度模型，其平滑可微的门控在预训练模型上稳定微调，无需大量重训练。在长文本推理任务中，最多节省15%算力，准确率仍保持在基线的90%以上；随着模型规模增大，该权衡显著改善。在指令微调模型上，全算力下实现准确率提升，近50%算力节省时达到基线质量。学习到的门控机制揭示了Transformer的信息流动规律（如起始符作为锚点），且可轻松集成量化、剪枝和自推测解码。
原文摘要 · Abstract (English)
We introduce GateSkip, a simple residual-stream gating mechanism that enables token-wise layer skipping in decoder-only LMs. Each Attention/MLP branch is equipped with a sigmoid-linear gate that condenses the branch's output before it re-enters the residual stream. During inference we rank tokens by the gate values and skip low-importance ones using a per-layer budget. While early-exit or router-based Mixture-of-Depths models are known to be unstable and need extensive retraining, our smooth, differentiable gates fine-tune stably on top of pretrained models. On long-form reasoning, we save up to 15% compute while retaining over 90% of baseline accuracy. For increasingly larger models, this tradeoff improves drastically. On instruction-tuned models we see accuracy gains at full compute and match baseline quality near 50% savings. The learned gates give insight into transformer information flow (e.g., BOS tokens act as anchors), and the method combines easily with quantization, pruning, and self-speculative decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。