arXiv:2606.29223cs.LG2026-06

通过并行探索多层深度,提升大模型解码效率。

Depth Exploration for LLM Decoding

论文配图:Depth Exploration for LLM Decoding
图 1 · 摘自论文原文
  • 在多个深度并行探索候选输出,而非单一深度选择。
  • 相比基线方法,推理速度提升显著,端到端吞吐量更优。
  • 适合追求高效推理的部署场景,尤其对多层模型有效。

自回归大模型解码在每一步都需经过完整的网络层级,尽管许多生成的词元在中间层已可预测。现有无损深度自适应方法通过选择单个非最终层出口验证预测,但测量显示该策略仍有优化空间:过晚选择浪费计算,过早则引发回退并丢弃依赖的草稿。本文提出深度探索解码(DEX),用并行探索多个候选深度替代单层选择。在每个提交位置,DEX 以最终层参考验证候选,仅提交最终层词元,并压缩探索结构以保留可复用状态分支。该展开-提交-压缩流程保持与标准自回归解码等价,同时降低每步提交成本。在训练有早期退出机制和标准大模型上,DEX 均优于代表性深度选择基线,且端到端吞吐量媲美推测解码与分布式解码方法。此外,随着探索深度细化,性能持续提升,表明并行深度探索是利用大模型解码中被低估的深度维度的可扩展方案。

原文摘要 · Abstract (English)

Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths. Existing lossless depth-adaptive methods exploit this redundancy by choosing a single non-final exit depth and verifying its prediction with the final-depth model. However, our measurements show that this selection-based strategy leaves substantial headroom: choosing an exit too late wastes computation, while choosing one too early triggers fallback and discards dependent drafts. We propose Depth Exploration Decoding (DEX), a lossless decoding algorithm that replaces single-depth selection with parallel exploration over multiple candidate depths. At each commit position, DEX validates candidates against the final-depth reference, commits exactly the final-depth token, and collapses the exploration lattice to retain only reusable branch states. This expand--commit--collapse procedure preserves equivalence to standard autoregressive decoding while reducing the cost of committing each token. Across early-exit-trained and standard LLMs, DEX outperforms representative depth-selection baselines and achieves competitive end-to-end throughput against speculative and distributed decoding methods. Moreover, DEX improves as the explored depths become finer, showing that parallel depth exploration provides a scalable way to exploit the underused depth axis of LLM decoding.

大模型推理深度探索高效解码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。