通过分层温度调节与自适应长度控制，提升自回归生成速度。
SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration
- 分层温度调节抑制早期预测的过度自信
- 根据词元难度动态调整推测长度，最高提速2.33倍
- 无需修改模型参数，适配多种大模型架构
自回归推理中的推测解码已成为加速大语言模型（LLM）推理的有前景方法。自草稿方法利用基础模型自身进行推测，避免了辅助草稿模型的开销，但存在局限：浅层常产生过度自信却错误的词元预测，且草稿序列中出现困难词元时会引发深层冗余计算，降低草稿接受率并削弱整体加速效果。为此，我们提出一种新颖的自草稿框架，通过在早期退出决策中采用分层温度退火抑制虚假置信度，并基于词元级解码难度自适应地约束推测长度。通过统一并行处理草稿词元的隐藏状态，该方法在保持与原始模型输出完全一致的同时最大化计算效率。无需修改基础模型参数，在多种长文本生成任务和多个模型架构上实现了最高达2.33倍的墙时速度提升。
原文摘要 · Abstract (English)
Speculative decoding has emerged as a promising approach to accelerate autoregressive inference in large language models (LLMs). Self-draft methods, which leverage the base LLM itself for speculation, avoid the overhead of auxiliary draft models but face limitations: shallow layers often produce overconfident yet incorrect token predictions, and the presence of difficult tokens in a draft sequence forces redundant computation through deeper layers, undermining both draft acceptance and overall speedup. To address these issues, we propose a novel self-draft framework that suppresses spurious confidence via layer-wise temperature annealing in early-exit decision and adaptively bounds speculation length based on token-wise decoding difficulty. By reprocessing the hidden states of draft tokens in a unified parallel pass through deep layers, our method maintains exact output equivalence with the original model while maximizing computational efficiency. It requires no modifications to the base LLM parameters and achieves up to 2.33x wall-time speedup over standard autoregressive decoding across diverse long-form generation tasks and multiple model architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。