现代大模型早期退出效果变差，作者提出评估新方法。
The Diminishing Returns of Early-Exit Decoding in Modern LLMs
- 提出量化模型早期退出潜力的新指标
- 发现新模型早期退出收益持续下降，200亿参数以上模型更优
- 适合关注推理加速与模型效率的研究者
在大语言模型推理中，早期退出通过在中间层停止计算来降低延迟和成本。然而，近年来大模型采用更优的预训练方法和架构，减少了层间冗余，可能限制早期退出机会。本文重新评估现代大模型中的分层早期退出，分析中间表示随训练的变化。提出一个量化模型内在早期退出潜力的指标，并构建基准供研究者探索不同模型与负载下的早期退出收益。结果表明，新世代模型的早期退出有效性呈下降趋势。此外，密集型Transformer相比Mixture-of-Experts和状态空间模型具有更高早期退出潜力；尤其超过200亿参数的模型及未经过特定调优的基础预训练模型表现更佳。
原文摘要 · Abstract (English)
In Large Language Model (LLM) inference, early-exit refers to stopping computation at an intermediate layer once the prediction is sufficiently confident, thereby reducing latency and cost. However, recent LLMs adopt improved pretraining recipes and architectures that reduce layer redundancy, potentially limiting early-exit opportunities. We re-evaluate layer-wise early-exit in modern LLMs and analyze how intermediate representations evolve during training. We introduce a metric to quantify a model's intrinsic suitability for early-exit and propose a benchmark for researchers to explore the potential early-exit benefits on different models and workloads. Our results show a diminishing trend in early-exit effectiveness across newer model generations. We further find that dense transformers generally offer greater early-exit potential than Mixture-of-Experts and State Space Models. In addition, larger models, particularly those with more than 20 billion parameters, and base pretrained models without specialized tuning tend to exhibit higher early-exit potential.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。