通过熵感知拒绝机制，提升推测解码的推理质量。
Entropy-Aware Token Rejection for Improving Speculative Decoding
- 基于令牌级熵检测不确定但一致的预测，拒绝低可信度输出。
- 在多个推理基准上超越标准推测解码和奖励引导变体，准确率提升显著。
- 无需训练、轻量级，适合追求高精度推理的场景。
推测解码（SD）通过轻量级草稿模型生成候选词，由更强的目标模型验证，以加速大语言模型推理。然而，标准SD主要聚焦加速，输出质量受限于目标模型。本文提出熵感知推测解码（EASD），一种无需训练的轻量级扩展方法，通过令牌级熵引导的拒绝机制提升推理质量。EASD检测草稿与目标模型均具高不确定性但顶层预测高度重合的情况，在此类不确定一致性情形下拒绝一致词并从目标分布重新采样，防止低置信度错误传播。在多个挑战性推理基准上的实验表明，EASD在保持相近推理效率的同时，持续提升准确率，甚至超越目标模型的独立性能，表明推测解码不仅能加速，还能有效提升推理质量。代码已公开于 https://github.com/ECNU-Text-Computing/EASD。
原文摘要 · Abstract (English)
Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronger target model to verify them. However, standard SD is mainly designed for acceleration, and its output quality is typically constrained by the target model. In this work, we propose Entropy-Aware Speculative Decoding (EASD), a lightweight and training-free extension of SD that improves reasoning quality through token-level entropy-guided rejection. EASD detects cases where both draft and target models exhibit high uncertainty while strongly overlapping in their top predictions. In such uncertain-agreement cases, EASD rejects the aligned token and resamples from the target distribution, preventing low-confidence errors from propagating. Experiments on challenging reasoning benchmarks show that EASD consistently improves accuracy over standard SD and reward-guided variants while maintaining comparable inference efficiency. Notably, EASD can surpass the standalone performance of the target model, suggesting that speculative decoding can serve not only as an acceleration method but also as an effective mechanism for improving reasoning quality. The code is available at https://github.com/ECNU-Text-Computing/EASD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。