通过频率引导选词提升大模型推理速度，减少误拒。
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
- 用历史拒绝记录找常见词形偏差，作为候选补救。
- 用概率比判断语义一致，而非严格匹配字面。
- 不需训练，提速最高2.33倍，适合实际部署。
推测解码通过让草稿标记跳过完整验证来加速自回归生成，但传统框架常因草稿模型产生语义正确但词汇不同的输出而频繁误拒。本文提出无需训练的校准推测解码（CSD），恢复被标准验证丢弃的有效标记。其核心为“频率引导候选选择与概率防护接受”原则，引入两个轻量模块：在线修正记忆，聚合历史拒绝信息，提出重复出现的词形偏差模式作为救援候选；语义一致性门控，使用概率比而非精确匹配验证候选有效性。在多种大型语言模型上的评估显示，CSD优于现有方法，峰值吞吐速度提升达2.33倍，同时保持所有任务的模型精度，并进一步提升复杂推理数据集表现。结果表明，CSD是高效且轻量的实用大模型部署方案。
原文摘要 · Abstract (English)
Speculative decoding accelerates autoregressive generation by letting draft tokens bypass full verification, but conventional frameworks suffer from frequent false rejections, particularly when draft models produce semantically correct but lexically divergent outputs. In this paper, we present Calibrated Speculative Decoding (CSD), a training-free framework that recovers valid tokens discarded by standard verification. Guided by the principle of "Frequency-Guided Candidate Selection and Probability-Guarded Acceptance," CSD incorporates two lightweight modules: Online Correction Memory, which aggregates historical rejections to propose recurring divergence patterns as rescue candidates, and Semantic Consistency Gating, which verifies candidate admissibility using probability ratios instead of exact token matching. Our evaluation across diverse large language models demonstrates that CSD outperforms existing methods, achieving a peak throughput speedup of 2.33x. CSD preserves model accuracy across all tasks while further boosting performance on complex reasoning datasets. These results establish CSD as a highly effective, lightweight solution for practical LLM deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。