不靠训练，用KL散度实现更可靠的LLM推理加速
Revisiting Judge Decoding from First Principles via Training-Free Distributional Divergence
- 基于KL散度构建无需训练的验证机制
- 在多个基准上性能媲美甚至超越复杂训练模型
- 对领域变化更鲁棒，彻底摆脱标注依赖
Judge Decoding 通过放宽推测解码的严格验证来加速大语言模型推理，但通常依赖昂贵且嘈杂的监督信号。本文从基本原理重新审视该范式，发现通过高成本监督学习得到的「关键性」评分，本质上已编码在草稿-目标分布的差异中。理论证明，学习到的线性判别器与KL散度存在结构对应关系，二者依赖相同的对数几率原始特征。基于此，我们提出一种简单、无需训练的验证机制，仅依赖KL散度。在推理与编程基准上的大量实验表明，该方法性能匹配或超越复杂训练模型（如AutoJudge），对领域偏移更具鲁棒性，并完全消除监督瓶颈。
原文摘要 · Abstract (English)
Judge Decoding accelerates LLM inference by relaxing the strict verification of Speculative Decoding, yet it typically relies on expensive and noisy supervision. In this work, we revisit this paradigm from first principles, revealing that the ``criticality'' scores learned via costly supervision are intrinsically encoded in the draft-target distributional divergence. We theoretically prove a structural correspondence between learned linear judges and Kullback-Leibler (KL) divergence, demonstrating they rely on the same underlying logit primitives. Guided by this, we propose a simple, training-free verification mechanism based on KL divergence. Extensive experiments across reasoning and coding benchmarks show that our method matches or outperforms complex trained judges (e.g., AutoJudge), offering superior robustness to domain shifts and eliminating the supervision bottleneck entirely.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。